[ 0.000000] Linux version 4.18.0rh8.10-debug (green@maintenance) (gcc version 8.5.0 20210514 (Red Hat 8.5.0-26) (GCC)) #2 SMP Mon Jul 14 01:24:22 EDT 2025 [ 0.000000] Command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers' [ 0.000000] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256 [ 0.000000] x86/fpu: Enabled xstate features 0x7, context size is 832 bytes, using 'standard' format. [ 0.000000] signal: max sigframe size: 1776 [ 0.000000] BIOS-provided physical RAM map: [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x00000000bffcdfff] usable [ 0.000000] BIOS-e820: [mem 0x00000000bffce000-0x00000000bfffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x0000000146dfffff] usable [ 0.000000] NX (Execute Disable) protection: active [ 0.000000] SMBIOS 2.8 present. [ 0.000000] DMI: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-8.fc42 06/10/2025 [ 0.000000] Hypervisor detected: KVM [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00 [ 0.000000] kvm-clock: using sched offset of 481503729 cycles [ 0.000000] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns [ 0.000000] tsc: Detected 2399.998 MHz processor [ 0.000000] last_pfn = 0x146e00 max_arch_pfn = 0x400000000 [ 0.000000] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT [ 0.000000] last_pfn = 0xbffce max_arch_pfn = 0x400000000 [ 0.000000] found SMP MP-table at [mem 0x000f54b0-0x000f54bf] [ 0.000000] RAMDISK: [mem 0xbcc54000-0xbffbffff] [ 0.000000] ACPI: Early table checksum verification disabled [ 0.000000] ACPI: RSDP 0x00000000000F52D0 000014 (v00 BOCHS ) [ 0.000000] ACPI: RSDT 0x00000000BFFE2439 000034 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACP 0x00000000BFFE22D5 000074 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: DSDT 0x00000000BFFE0040 002295 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACS 0x00000000BFFE0000 000040 [ 0.000000] ACPI: APIC 0x00000000BFFE2349 000090 (v03 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: HPET 0x00000000BFFE23D9 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: WAET 0x00000000BFFE2411 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: Reserving FACP table memory at [mem 0xbffe22d5-0xbffe2348] [ 0.000000] ACPI: Reserving DSDT table memory at [mem 0xbffe0040-0xbffe22d4] [ 0.000000] ACPI: Reserving FACS table memory at [mem 0xbffe0000-0xbffe003f] [ 0.000000] ACPI: Reserving APIC table memory at [mem 0xbffe2349-0xbffe23d8] [ 0.000000] ACPI: Reserving HPET table memory at [mem 0xbffe23d9-0xbffe2410] [ 0.000000] ACPI: Reserving WAET table memory at [mem 0xbffe2411-0xbffe2438] [ 0.000000] No NUMA configuration found [ 0.000000] Faking a node at [mem 0x0000000000000000-0x0000000146dfffff] [ 0.000000] NODE_DATA(0) allocated [mem 0x1465a3000-0x1465cdfff] [ 0.000000] Reserving 256MB of memory at 2752MB for crashkernel (System RAM: 4205MB) [ 0.000000] Zone ranges: [ 0.000000] DMA [mem 0x0000000000001000-0x0000000000ffffff] [ 0.000000] DMA32 [mem 0x0000000001000000-0x00000000ffffffff] [ 0.000000] Normal [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Device empty [ 0.000000] Movable zone start for each node [ 0.000000] Early memory node ranges [ 0.000000] node 0: [mem 0x0000000000001000-0x000000000009efff] [ 0.000000] node 0: [mem 0x0000000000100000-0x00000000bffcdfff] [ 0.000000] node 0: [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Zeroed struct page in unavailable ranges: 4756 pages [ 0.000000] Initmem setup node 0 [mem 0x0000000000001000-0x0000000146dfffff] [ 0.000000] ACPI: PM-Timer IO Port: 0x608 [ 0.000000] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1]) [ 0.000000] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23 [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level) [ 0.000000] Using ACPI (MADT) for SMP configuration information [ 0.000000] ACPI: HPET id: 0x8086a201 base: 0xfed00000 [ 0.000000] TSC deadline timer available [ 0.000000] smpboot: Allowing 4 CPUs, 0 hotplug CPUs [ 0.000000] kvm-guest: KVM setup pv remote TLB flush [ 0.000000] kvm-guest: setup PV sched yield [ 0.000000] PM: Registered nosave memory: [mem 0x00000000-0x00000fff] [ 0.000000] PM: Registered nosave memory: [mem 0x0009f000-0x0009ffff] [ 0.000000] PM: Registered nosave memory: [mem 0x000a0000-0x000effff] [ 0.000000] PM: Registered nosave memory: [mem 0x000f0000-0x000fffff] [ 0.000000] PM: Registered nosave memory: [mem 0xbffce000-0xbfffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xc0000000-0xfeffbfff] [ 0.000000] PM: Registered nosave memory: [mem 0xfeffc000-0xfeffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xff000000-0xfffbffff] [ 0.000000] PM: Registered nosave memory: [mem 0xfffc0000-0xffffffff] [ 0.000000] [mem 0xc0000000-0xfeffbfff] available for PCI devices [ 0.000000] Booting paravirtualized kernel on KVM [ 0.000000] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns [ 0.000000] setup_percpu: NR_CPUS:8192 nr_cpumask_bits:4 nr_cpu_ids:4 nr_node_ids:1 [ 0.000000] percpu: Embedded 63 pages/cpu s221184 r8192 d28672 u524288 [ 0.000000] kvm-guest: PV spinlocks enabled [ 0.000000] PV qspinlock hash table entries: 256 (order: 0, 4096 bytes, linear) [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 1059606 [ 0.000000] Policy zone: Normal [ 0.000000] Kernel command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] Specific versions of hardware are certified with Red Hat Enterprise Linux 8. Please see the list of hardware certified with Red Hat Enterprise Linux 8 at https://catalog.redhat.com. [ 0.000000] audit: disabled (until reboot) [ 0.000000] software IO TLB: area num 4. [ 0.000000] Memory: 2829652K/4306352K available (18435K kernel code, 11221K rwdata, 7248K rodata, 2908K init, 18040K bss, 524584K reserved, 0K cma-reserved) [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1 [ 0.000000] kmemleak: Kernel memory leak detector disabled [ 0.000000] ftrace: allocating 41240 entries in 162 pages [ 0.000000] ftrace: allocated 162 pages with 3 groups [ 0.000000] rcu: Hierarchical RCU implementation. [ 0.000000] rcu: RCU event tracing is enabled. [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=8192 to nr_cpu_ids=4. [ 0.000000] rcu: RCU callback double-/use-after-free debug enabled. [ 0.000000] Rude variant of Tasks RCU enabled. [ 0.000000] Tracing variant of Tasks RCU enabled. [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies. [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4 [ 0.000000] NR_IRQS: 524544, nr_irqs: 456, preallocated irqs: 16 [ 0.000000] random: get_random_bytes called from start_kernel+0x622/0x9a8 with crng_init=0 [ 0.001000] Console: colour *CGA 80x25 [ 0.001000] printk: console [ttyS1] enabled [ 0.001000] ACPI: Core revision 20220331 [ 0.001000] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns [ 0.001011] APIC: Switch to symmetric I/O mode setup [ 0.003384] x2apic enabled [ 0.004008] Switched APIC routing to physical x2apic. [ 0.005016] kvm-guest: setup PV IPIs [ 0.008391] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1 [ 0.009000] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 0.009020] Calibrating delay loop (skipped) preset value.. 4799.99 BogoMIPS (lpj=2399998) [ 0.010015] pid_max: default: 32768 minimum: 301 [ 0.012073] LSM: Security Framework initializing [ 0.013059] Yama: becoming mindful. [ 0.014044] SELinux: Initializing. [ 0.015104] *** VALIDATE selinux *** [ 0.023661] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, vmalloc) [ 0.027605] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, vmalloc) [ 0.028151] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.029109] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.030117] *** VALIDATE tmpfs *** [ 0.031480] *** VALIDATE proc *** [ 0.032228] *** VALIDATE cgroup *** [ 0.033011] *** VALIDATE cgroup2 *** [ 0.034285] x86/cpu: User Mode Instruction Prevention (UMIP) activated [ 0.035161] Last level iTLB entries: 4KB 0, 2MB 0, 4MB 0 [ 0.036010] Last level dTLB entries: 4KB 0, 2MB 0, 4MB 0, 1GB 0 [ 0.037029] Spectre V2 : User space: Vulnerable [ 0.038009] Speculative Store Bypass: Vulnerable [ 0.040943] debug: unmapping init [mem 0xffffffffa8e59000-0xffffffffa8e60fff] [ 0.042179] smpboot: CPU0: Intel(R) Xeon(R) CPU E5-2695 v2 @ 2.40GHz (family: 0x6, model: 0x3e, stepping: 0x4) [ 0.043699] Performance Events: IvyBridge events, full-width counters, Intel PMU driver. [ 0.044026] ... version: 2 [ 0.045011] ... bit width: 48 [ 0.046010] ... generic registers: 4 [ 0.047012] ... value mask: 0000ffffffffffff [ 0.048013] ... max period: 00007fffffffffff [ 0.049013] ... fixed-purpose events: 3 [ 0.050011] ... event mask: 000000070000000f [ 0.052212] rcu: Hierarchical SRCU implementation. [ 0.054728] smp: Bringing up secondary CPUs ... [ 0.055489] x86: Booting SMP configuration: [ 0.056019] .... node #0, CPUs: #1 #2 #3 [ 0.059010] smp: Brought up 1 node, 4 CPUs [ 0.061009] smpboot: Max logical packages: 1 [ 0.062015] smpboot: Total of 4 processors activated (19199.98 BogoMIPS) [ 0.144954] node 0 deferred pages initialised in 80ms [ 0.151008] devtmpfs: initialized [ 0.152175] x86/mm: Memory block size: 128MB [ 0.154672] gcov: version magic: 0x41383552 [ 0.155618] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns [ 0.157079] futex hash table entries: 1024 (order: 4, 65536 bytes, vmalloc) [ 0.158238] pinctrl core: initialized pinctrl subsystem [ 0.159187] [ 0.159913] ************************************************************* [ 0.160015] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.161011] ** ** [ 0.162015] ** IOMMU DebugFS SUPPORT HAS BEEN ENABLED IN THIS KERNEL ** [ 0.163014] ** ** [ 0.164015] ** This means that this kernel is built to expose internal ** [ 0.165011] ** IOMMU data structures, which may compromise security on ** [ 0.166012] ** your system. ** [ 0.167014] ** ** [ 0.168014] ** If you see this message and you are not debugging the ** [ 0.169027] ** kernel, report this immediately to your vendor! ** [ 0.170018] ** ** [ 0.171015] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.172017] ************************************************************* [ 0.173732] NET: Registered protocol family 16 [ 0.174453] DMA: preallocated 512 KiB GFP_KERNEL pool for atomic allocations [ 0.175064] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations [ 0.176061] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations [ 0.177457] cpuidle: using governor menu [ 0.179989] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5 [ 0.182531] PCI: Using configuration type 1 for base access [ 0.184124] core: PMU erratum BJ122, BV98, HSD29 worked around, HT is on [ 0.193057] HugeTLB registered 1.00 GiB page size, pre-allocated 0 pages [ 0.194023] HugeTLB registered 2.00 MiB page size, pre-allocated 0 pages [ 0.195182] cryptd: max_cpu_qlen set to 1000 [ 0.197149] ACPI: Added _OSI(Module Device) [ 0.198014] ACPI: Added _OSI(Processor Device) [ 0.199021] ACPI: Added _OSI(3.0 _SCP Extensions) [ 0.200013] ACPI: Added _OSI(Processor Aggregator Device) [ 0.203785] ACPI: 1 ACPI AML tables successfully acquired and loaded [ 0.206713] ACPI: Interpreter enabled [ 0.207066] ACPI: PM: (supports S0 S3 S4 S5) [ 0.208012] ACPI: Using IOAPIC for interrupt routing [ 0.209113] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug [ 0.210396] ACPI: Enabled 2 GPEs in block 00 to 0F [ 0.220540] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff]) [ 0.221064] acpi PNP0A03:00: _OSC: OS supports [ASPM ClockPM Segments MSI HPX-Type3] [ 0.222021] acpi PNP0A03:00: _OSC: not requesting OS control; OS requires [ExtendedConfig ASPM ClockPM MSI] [ 0.223082] acpi PNP0A03:00: fail to add MMCONFIG information, can't access extended PCI configuration space under this bridge. [ 0.225343] acpiphp: Slot [2] registered [ 0.226177] acpiphp: Slot [5] registered [ 0.227210] acpiphp: Slot [6] registered [ 0.228168] acpiphp: Slot [7] registered [ 0.229122] acpiphp: Slot [8] registered [ 0.230108] acpiphp: Slot [9] registered [ 0.231134] acpiphp: Slot [10] registered [ 0.232131] acpiphp: Slot [3] registered [ 0.233094] acpiphp: Slot [4] registered [ 0.234099] acpiphp: Slot [11] registered [ 0.235093] acpiphp: Slot [12] registered [ 0.236104] acpiphp: Slot [13] registered [ 0.237106] acpiphp: Slot [14] registered [ 0.238093] acpiphp: Slot [15] registered [ 0.239117] acpiphp: Slot [16] registered [ 0.240096] acpiphp: Slot [17] registered [ 0.241072] acpiphp: Slot [18] registered [ 0.242099] acpiphp: Slot [19] registered [ 0.243103] acpiphp: Slot [20] registered [ 0.244113] acpiphp: Slot [21] registered [ 0.245130] acpiphp: Slot [22] registered [ 0.246111] acpiphp: Slot [23] registered [ 0.247103] acpiphp: Slot [24] registered [ 0.248109] acpiphp: Slot [25] registered [ 0.249126] acpiphp: Slot [26] registered [ 0.250105] acpiphp: Slot [27] registered [ 0.251098] acpiphp: Slot [28] registered [ 0.252126] acpiphp: Slot [29] registered [ 0.253121] acpiphp: Slot [30] registered [ 0.254121] acpiphp: Slot [31] registered [ 0.255103] PCI host bridge to bus 0000:00 [ 0.256027] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window] [ 0.259028] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window] [ 0.261024] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window] [ 0.264026] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window] [ 0.267023] pci_bus 0000:00: root bus resource [mem 0xe0000000000-0xe007fffffff window] [ 0.270041] pci_bus 0000:00: root bus resource [bus 00-ff] [ 0.272188] pci 0000:00:00.0: [8086:1237] type 00 class 0x060000 [ 0.275128] pci 0000:00:01.0: [8086:7000] type 00 class 0x060100 [ 0.278392] pci 0000:00:01.1: [8086:7010] type 00 class 0x010180 [ 0.289011] pci 0000:00:01.1: reg 0x20: [io 0xc320-0xc32f] [ 0.295316] pci 0000:00:01.1: legacy IDE quirk: reg 0x10: [io 0x01f0-0x01f7] [ 0.298019] pci 0000:00:01.1: legacy IDE quirk: reg 0x14: [io 0x03f6] [ 0.300013] pci 0000:00:01.1: legacy IDE quirk: reg 0x18: [io 0x0170-0x0177] [ 0.303018] pci 0000:00:01.1: legacy IDE quirk: reg 0x1c: [io 0x0376] [ 0.306331] pci 0000:00:01.3: [8086:7113] type 00 class 0x068000 [ 0.309771] pci 0000:00:01.3: quirk: [io 0x0600-0x063f] claimed by PIIX4 ACPI [ 0.313046] pci 0000:00:01.3: quirk: [io 0x0700-0x070f] claimed by PIIX4 SMB [ 0.316860] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 [ 0.323017] pci 0000:00:02.0: reg 0x10: [io 0xc300-0xc31f] [ 0.339024] pci 0000:00:02.0: reg 0x20: [mem 0xe0000000000-0xe0000003fff 64bit pref] [ 0.343017] pci 0000:00:02.0: reg 0x30: [mem 0xfeb80000-0xfebbffff pref] [ 0.349149] pci 0000:00:05.0: [1af4:1001] type 00 class 0x010000 [ 0.357014] pci 0000:00:05.0: reg 0x10: [io 0xc000-0xc07f] [ 0.365015] pci 0000:00:05.0: reg 0x14: [mem 0xfebc0000-0xfebc0fff] [ 0.385014] pci 0000:00:05.0: reg 0x20: [mem 0xe0000004000-0xe0000007fff 64bit pref] [ 0.395314] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 [ 0.412018] pci 0000:00:06.0: reg 0x10: [io 0xc080-0xc0ff] [ 0.421023] pci 0000:00:06.0: reg 0x14: [mem 0xfebc1000-0xfebc1fff] [ 0.437015] pci 0000:00:06.0: reg 0x20: [mem 0xe0000008000-0xe000000bfff 64bit pref] [ 0.450881] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 [ 0.458015] pci 0000:00:07.0: reg 0x10: [io 0xc100-0xc17f] [ 0.465018] pci 0000:00:07.0: reg 0x14: [mem 0xfebc2000-0xfebc2fff] [ 0.483017] pci 0000:00:07.0: reg 0x20: [mem 0xe000000c000-0xe000000ffff 64bit pref] [ 0.500145] pci 0000:00:08.0: [1af4:1001] type 00 class 0x010000 [ 0.511025] pci 0000:00:08.0: reg 0x10: [io 0xc180-0xc1ff] [ 0.517015] pci 0000:00:08.0: reg 0x14: [mem 0xfebc3000-0xfebc3fff] [ 0.533016] pci 0000:00:08.0: reg 0x20: [mem 0xe0000010000-0xe0000013fff 64bit pref] [ 0.543543] pci 0000:00:09.0: [1af4:1001] type 00 class 0x010000 [ 0.550020] pci 0000:00:09.0: reg 0x10: [io 0xc200-0xc27f] [ 0.559016] pci 0000:00:09.0: reg 0x14: [mem 0xfebc4000-0xfebc4fff] [ 0.575019] pci 0000:00:09.0: reg 0x20: [mem 0xe0000014000-0xe0000017fff 64bit pref] [ 0.587174] pci 0000:00:0a.0: [1af4:1001] type 00 class 0x010000 [ 0.594021] pci 0000:00:0a.0: reg 0x10: [io 0xc280-0xc2ff] [ 0.600016] pci 0000:00:0a.0: reg 0x14: [mem 0xfebc5000-0xfebc5fff] [ 0.617016] pci 0000:00:0a.0: reg 0x20: [mem 0xe0000018000-0xe000001bfff 64bit pref] [ 0.626415] ACPI: PCI: Interrupt link LNKA configured for IRQ 10 [ 0.629402] ACPI: PCI: Interrupt link LNKB configured for IRQ 10 [ 0.632377] ACPI: PCI: Interrupt link LNKC configured for IRQ 11 [ 0.635373] ACPI: PCI: Interrupt link LNKD configured for IRQ 11 [ 0.637295] ACPI: PCI: Interrupt link LNKS configured for IRQ 9 [ 0.642029] iommu: Default domain type: Passthrough [ 0.644380] SCSI subsystem initialized [ 0.645140] ACPI: bus type USB registered [ 0.647104] usbcore: registered new interface driver usbfs [ 0.649081] usbcore: registered new interface driver hub [ 0.650078] usbcore: registered new device driver usb [ 0.652140] pps_core: LinuxPPS API ver. 1 registered [ 0.654011] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti [ 0.657057] PTP clock support registered [ 0.659120] EDAC MC: Ver: 3.0.0 [ 0.660323] PCI: Using ACPI for IRQ routing [ 0.663026] NetLabel: Initializing [ 0.664009] NetLabel: domain hash size = 128 [ 0.665009] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO [ 0.667092] NetLabel: unlabeled traffic allowed by default [ 0.670042] vgaarb: loaded [ 0.671242] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0 [ 0.673010] hpet0: 3 comparators, 64-bit 100.000000 MHz counter [ 0.678307] clocksource: Switched to clocksource kvm-clock [ 0.784763] VFS: Disk quotas dquot_6.6.0 [ 0.786210] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) [ 0.788843] *** VALIDATE ramfs *** [ 0.790765] *** VALIDATE hugetlbfs *** [ 0.792638] pnp: PnP ACPI init [ 0.796739] pnp: PnP ACPI: found 6 devices [ 0.813844] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns [ 0.817084] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window] [ 0.819086] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window] [ 0.820823] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window] [ 0.822986] pci_bus 0000:00: resource 7 [mem 0xc0000000-0xfebfffff window] [ 0.825239] pci_bus 0000:00: resource 8 [mem 0xe0000000000-0xe007fffffff window] [ 0.828239] NET: Registered protocol family 2 [ 0.830710] IP idents hash table entries: 131072 (order: 8, 1048576 bytes, vmalloc) [ 0.836066] tcp_listen_portaddr_hash hash table entries: 4096 (order: 5, 163840 bytes, vmalloc) [ 0.840100] TCP established hash table entries: 65536 (order: 7, 524288 bytes, vmalloc) [ 0.846246] TCP bind hash table entries: 65536 (order: 9, 2097152 bytes, vmalloc) [ 0.850771] TCP: Hash tables configured (established 65536 bind 65536) [ 0.854215] MPTCP token hash table entries: 8192 (order: 6, 393216 bytes, vmalloc) [ 0.858356] UDP hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 0.861822] UDP-Lite hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 0.864895] NET: Registered protocol family 1 [ 0.868100] RPC: Registered named UNIX socket transport module. [ 0.869759] RPC: Registered udp transport module. [ 0.871169] RPC: Registered tcp transport module. [ 0.872772] RPC: Registered tcp NFSv4.1 backchannel transport module. [ 0.875311] NET: Registered protocol family 44 [ 0.876934] pci 0000:00:00.0: Limiting direct PCI/PCI transfers [ 0.879362] pci 0000:00:01.0: PIIX3: Enabling Passive Release [ 0.881880] pci 0000:00:01.0: Activating ISA DMA hang workarounds [ 0.885363] PCI: CLS 0 bytes, default 64 [ 0.886883] Unpacking initramfs... [ 3.163712] debug: unmapping init [mem 0xffff952cfcc54000-0xffff952cfffbffff] [ 3.184081] PCI-DMA: Using software bounce buffering for IO (SWIOTLB) [ 3.190549] software IO TLB: mapped [mem 0x00000000a8000000-0x00000000ac000000] (64MB) [ 3.197889] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 5.142897] Initialise system trusted keyrings [ 5.145068] Key type blacklist registered [ 5.147538] workingset: timestamp_bits=36 max_order=20 bucket_order=0 [ 5.159812] zbud: loaded [ 5.163465] *** VALIDATE nfs *** [ 5.165134] *** VALIDATE nfs4 *** [ 5.166863] pstore: using deflate compression [ 5.171230] Platform Keyring initialized [ 5.497949] NET: Registered protocol family 38 [ 5.501258] Key type asymmetric registered [ 5.503462] Asymmetric key parser 'x509' registered [ 5.507420] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 247) [ 5.512396] io scheduler mq-deadline registered [ 5.515618] io scheduler kyber registered [ 5.518726] io scheduler bfq registered [ 5.521845] atomic64_test: passed for x86-64 platform with CX8 and with SSE [ 5.528428] shpchp: Standard Hot Plug PCI Controller Driver version: 0.4 [ 5.535855] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0 [ 5.542487] ACPI: Power Button [PWRF] [ 5.555427] ACPI: \_SB_.LNKB: Enabled at IRQ 10 [ 5.567159] ACPI: \_SB_.LNKA: Enabled at IRQ 11 [ 5.589758] ACPI: \_SB_.LNKC: Enabled at IRQ 11 [ 5.606471] ACPI: \_SB_.LNKD: Enabled at IRQ 10 [ 5.643767] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled [ 5.685073] 00:03: ttyS1 at I/O 0x2f8 (irq = 3, base_baud = 115200) is a 16550A [ 5.729533] 00:04: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A [ 5.755795] Non-volatile memory driver v1.3 [ 5.762604] Linux agpgart interface v0.103 [ 5.879920] virtio_blk virtio1: [vda] 134608 512-byte logical blocks (68.9 MB/65.7 MiB) [ 5.885688] vda: detected capacity change from 0 to 68919296 [ 5.941504] virtio_blk virtio2: [vdb] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) [ 5.957534] vdb: detected capacity change from 0 to 1073741824 [ 6.012582] virtio_blk virtio3: [vdc] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 6.031395] vdc: detected capacity change from 0 to 2621440000 [ 6.099172] virtio_blk virtio4: [vdd] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 6.104289] vdd: detected capacity change from 0 to 2621440000 [ 6.145741] virtio_blk virtio5: [vde] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 6.149572] vde: detected capacity change from 0 to 4294967296 [ 6.172057] virtio_blk virtio6: [vdf] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 6.179728] vdf: detected capacity change from 0 to 4294967296 [ 6.195882] libphy: Fixed MDIO Bus: probed [ 6.202644] usbcore: registered new interface driver usbserial_generic [ 6.205168] usbserial: USB Serial support registered for generic [ 6.216303] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12 [ 6.220898] serio: i8042 KBD port at 0x60,0x64 irq 1 [ 6.223553] serio: i8042 AUX port at 0x60,0x64 irq 12 [ 6.230721] mousedev: PS/2 mouse device common for all mice [ 6.234953] rtc_cmos 00:05: RTC can wake from S4 [ 6.242830] rtc_cmos 00:05: registered as rtc0 [ 6.245798] rtc_cmos 00:05: alarms up to one day, y3k, 242 bytes nvram, hpet irqs [ 6.257091] intel_pstate: CPU model not supported [ 6.266773] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input1 [ 6.271205] hid: raw HID events driver (C) Jiri Kosina [ 6.281261] usbcore: registered new interface driver usbhid [ 6.286314] usbhid: USB HID core driver [ 6.291337] drop_monitor: Initializing network drop monitor service [ 6.294735] Initializing XFRM netlink socket [ 6.295764] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input4 [ 6.302427] NET: Registered protocol family 10 [ 6.303712] Segment Routing with IPv6 [ 6.303755] NET: Registered protocol family 17 [ 6.304101] mpls_gso: MPLS GSO support [ 6.326164] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input3 [ 6.331797] RAS: Correctable Errors collector initialized. [ 6.334808] AVX version of gcm_enc/dec engaged. [ 6.336854] AES CTR mode by8 optimization enabled [ 6.474765] sched_clock: Marking stable (6474746447, 0)->(7500087783, -1025341336) [ 6.479766] registered taskstats version 1 [ 6.482761] Loading compiled-in X.509 certificates [ 6.486241] zswap: loaded using pool lzo/zbud [ 6.525889] Key type big_key registered [ 6.543789] Key type encrypted registered [ 6.545863] ima: No TPM chip found, activating TPM-bypass! [ 6.548422] ima: Allocated hash algorithm: sha1 [ 6.550437] ima: No architecture policies found [ 6.555877] evm: Initialising EVM extended attributes: [ 6.560281] evm: security.selinux [ 6.564839] evm: security.ima [ 6.567279] evm: security.capability [ 6.569825] evm: HMAC attrs: 0x1 [ 6.581195] rtc_cmos 00:05: setting system clock to 2026-04-29 20:17:59 UTC (1777493879) [ 6.602602] debug: unmapping init [mem 0xffffffffa9e03000-0xffffffffa9ffffff] [ 6.610887] debug: unmapping init [mem 0xffffffffa8b82000-0xffffffffa8e58fff] [ 6.622190] Write protecting the kernel read-only data: 28672k [ 6.633988] debug: unmapping init [mem 0xffffffffa7203000-0xffffffffa73fffff] [ 6.652620] debug: unmapping init [mem 0xffffffffa7b14000-0xffffffffa7bfffff] [ 6.777866] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 6.801648] systemd[1]: Detected virtualization kvm. [ 6.804666] systemd[1]: Detected architecture x86-64. [ 6.806975] systemd[1]: Running in initial RAM disk. Welcome to Rocky Linux 8.10 (Green Obsidian) dracut-049-233.git20240115.el8 (Initramfs)! [ 6.897578] systemd[1]: No hostname configured. [ 6.901444] systemd[1]: Set hostname to . [ 6.903832] random: systemd: uninitialized urandom read (16 bytes read) [ 6.906256] systemd[1]: Initializing machine ID from random generator. [ 7.010922] random: ln: uninitialized urandom read (6 bytes read) [ 7.203758] random: systemd: uninitialized urandom read (16 bytes read) [ 7.211917] systemd[1]: Listening on Journal Socket (/dev/log). [ OK ] Listening on Journal Socket (/dev/log). [ 7.237339] systemd[1]: Listening on udev Control Socket. [ OK ] Listening on udev Control Socket. [ 7.247152] systemd[1]: Reached target Initrd Root Device. [ OK ] Reached target Initrd Root Device. [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Reached target Local Encrypted Volumes. [ OK ] Reached target Timers. [ OK ] Reached target Paths. [ OK ] Reached target Slices. [ OK ] Reached target Local File Systems. [ OK ] Listening on Journal Socket. Starting Create list of required st…ce nodes for the current kernel... Starting Journal Service... Starting Create Volatile Files and Directories... Starting Apply Kernel Variables... [ OK ] Listening on udev Kernel Socket. [ OK ] Reached target Sockets. Starting Setup Virtual Console... [ OK ] Started Memstrack Anylazing Service. [ OK ] Reached target Swap. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Started Create Volatile Files and Directories. [ OK ] Started Apply Kernel Variables. [ OK ] Started Setup Virtual Console. Starting dracut cmdline hook... Starting Create Static Device Nodes in /dev... [ OK ] Started Create Static Device Nodes in /dev. [ OK ] Started Journal Service. [ OK ] Started dracut cmdline hook. Starting dracut pre-udev hook... [ 8.866497] device-mapper: uevent: version 1.0.3 [ 8.870722] device-mapper: ioctl: 4.46.0-ioctl (2022-02-22) initialised: dm-devel@redhat.com [ OK ] Started dracut pre-udev hook. Starting udev Kernel Device Manager... [ OK ] Started udev Kernel Device Manager. Starting dracut pre-trigger hook... [ OK ] Started dracut pre-trigger hook. Starting udev Coldplug all Devices... Mounting Kernel Configuration File System... [ OK ] Mounted Kernel Configuration File System. [ OK ] Started udev Coldplug all Devices. [ OK ] Reached target System Initialization. [ OK ] Reached target Basic System. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. Starting dracut initqueue hook... [ 10.752382] virtio_net virtio0 ens2: renamed from eth0 [ 10.903807] random: fast init done [ 10.988944] scsi host0: ata_piix [ 11.011121] scsi host1: ata_piix [ 11.015894] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc320 irq 14 [ 11.024992] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc328 irq 15 [ 16.092332] random: crng init done [ 16.094241] random: 7 urandom warning(s) missed due to ratelimiting [ 16.812729] dracut-initqueue[582]: RTNETLINK answers: File exists Starting nbd nbd0... [ OK ] Started nbd nbd0. [ OK ] Started dracut initqueue hook. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Mounting /sysroot... [ 19.039386] EXT4-fs (nbd0): mounted filesystem with ordered data mode. Opts: (null) [ OK ] Mounted /sysroot. [ OK ] Reached target Initrd Root File System. Starting Reload Configuration from the Real Root... [ OK ] Started Reload Configuration from the Real Root. [ OK ] Reached target Initrd File Systems. [ OK ] Reached target Initrd Default Target. Starting dracut pre-pivot and cleanup hook... [ OK ] Started dracut pre-pivot and cleanup hook. Starting Cleaning Up and Shutting Down Daemons... [ OK ] Stopped target Timers. Stopping Hardware RNG Entropy Gatherer Daemon... [ OK ] Stopped dracut pre-pivot and cleanup hook. [ OK ] Stopped target Initrd Default Target. [ OK ] Stopped target Initrd Root Device. [ OK ] Stopped target Remote File Systems. [ OK ] Stopped target Remote File Systems (Pre). [ OK ] Stopped dracut initqueue hook. [ OK ] Stopped Hardware RNG Entropy Gatherer Daemon. [ OK ] Stopped target Basic System. [ OK ] Stopped target Paths. [ OK ] Stopped target Slices. [ OK ] Stopped target System Initialization. [ OK ] Stopped Apply Kernel Variables. [ OK ] Stopped target Local Encrypted Volumes. [ OK ] Stopped Dispatch Password Requests to Console Directory Watch. [ OK ] Stopped udev Coldplug all Devices. [ OK ] Stopped dracut pre-trigger hook. Stopping udev Kernel Device Manager... [ OK ] Stopped Create Volatile Files and Directories. [ OK ] Stopped target Local File Systems. [ OK ] Stopped target Swap. [ OK ] Stopped target Sockets. [ OK ] Stopped udev Kernel Device Manager. [ OK ] Started Cleaning Up and Shutting Down Daemons. [ OK ] Stopped Create Static Device Nodes in /dev. [ OK ] Stopped Create list of required sta…vice nodes for the current kernel. [ OK ] Stopped dracut pre-udev hook. [ OK ] Stopped dracut cmdline hook. [ OK ] Closed udev Control Socket. [ OK ] Closed udev Kernel Socket. Starting Cleanup udevd DB... [ OK ] Started Cleanup udevd DB. [ OK ] Reached target Switch Root. Starting Switch Root... [ 22.094249] printk: systemd: 26 output lines suppressed due to ratelimiting [ 23.001954] SELinux: Disabled at runtime. [ 23.108658] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 23.124351] systemd[1]: Detected virtualization kvm. [ 23.127734] systemd[1]: Detected architecture x86-64. Welcome to Rocky Linux 8.10 (Green Obsidian)! [ 25.010226] systemd[1]: initrd-switch-root.service: Succeeded. [ 25.014599] systemd[1]: Stopped Switch Root. [ OK ] Stopped Switch Root. [ 25.023373] systemd[1]: systemd-journald.service: Service has no hold-off time (RestartSec=0), scheduling restart. [ 25.030112] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. [ 25.041352] systemd[1]: Stopped Journal Service. [ OK ] Stopped Journal Service. [ 25.057911] systemd[1]: Starting Journal Service... Starting Journal Service... [ 25.084055] systemd[1]: Activating swap /dev/disk/by-label/SWAP... Activating swap /dev/disk/by-label/SWAP... [ OK ] Created slice system-sshd\x2dkeygen.slice. [ OK ] Created slice system-serial\x2dgetty.slice. [ OK ] Listening on udev Kernel Socket. Mounting POSIX Message Queue File System... [ 25.183788] Adding 1048572k swap on /dev/vdb. Priority:-2 extents:1 across:1048572k FS [ OK ] Created slice system-getty.slice. Mounting Huge Pages File System... [ OK ] Listening on initctl Compatibility Named Pipe. [ OK ] Stopped target Switch Root. [ OK ] Stopped target Initrd File Systems. Mounting Kernel Debug File System... Starting Create list of required st…ce nodes for the current kernel... [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Stopped target Initrd Root File System. [ OK ] Listening on Process Core Dump Socket. [ OK ] Listening on udev Control Socket. Starting udev Coldplug all Devices... [FAILED] Failed to set up automount Arbitrar…rmats File System Automount Point. See 'systemctl status proc-sys-fs-binfmt_misc.automount' for details. [ OK ] Started Forward Password Requests to Wall Directory Watch. [ OK ] Reached target Paths. [ OK ] Reached target rpc_pipefs.target. Starting Apply Kernel Variables... Starting Remount Root and Kernel File Systems... [ OK ] Created slice User and Session Slice. [ OK ] Reached target Slices. [ OK ] Reached target Local Encrypted Volumes. [ OK ] Listening on RPCbind Server Activation Socket. [ OK ] Reached target RPC Port Mapper. [ OK ] Started Journal Service. [ OK ] Activated swap /dev/disk/by-label/SWAP. [ OK ] Mounted POSIX Message Queue File System. [ OK ] Mounted Huge Pages File System. [ OK ] Mounted Kernel Debug File System. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Started Apply Kernel Variables. [FAILED] Failed to start Remount Root and Kernel File Systems. See 'systemctl status systemd-remount-fs.service' for details. Starting Configure read-only root support... Starting Create Static Device Nodes in /dev... [ OK ] Reached target Swap. Starting Flush Journal to Persistent Storage... [ OK ] Started Flush Journal to Persistent Storage. [ OK ] Started udev Coldplug all Devices. [ OK ] Started Create Static Device Nodes in /dev. Starting udev Kernel Device Manager... [ OK ] Reached target Local File Systems (Pre). Mounting /home/green/git/lustre-release... Mounting /mnt... [ OK ] Mounted /mnt. [ 26.699268] squashfs: version 4.0 (2009/01/31) Phillip Lougher [ OK ] Mounted /home/green/git/lustre-release. [ OK ] Started udev Kernel Device Manager. [ 27.864681] input: PC Speaker as /devices/platform/pcspkr/input/input5 [ 28.022532] piix4_smbus 0000:00:01.3: SMBus Host Controller at 0x700, revision 0 [ 28.359663] RAPL PMU: API unit is 2^-32 Joules, 0 fixed counters, 10737418240 ms ovfl timer [ 28.428611] EDAC sbridge: Ver: 1.1.2 [* ] A start job is running for Configur…-only root support (7s / no limit) [** ] A start job is running for Configur…-only root support (8s / no limit)[ 33.609510] Key type dns_resolver registered [*** ] A start job is running for Configur…-only root support (8s / no limit) [ *** ] A start job is running for Configur…-only root support (9s / no limit)[ 34.417527] NFS: Registering the id_resolver key type [ 34.424600] Key type id_resolver registered [ 34.429934] Key type id_legacy registered [ *** ] A start job is running for Configur…-only root support (9s / no limit) [ OK ] Started Configure read-only root support. [ OK ] Reached target Local File Systems. Starting Mark the need to relabel after reboot... Starting Rebuild Dynamic Linker Cache... Starting Load/Save Random Seed... Starting Create Volatile Files and Directories... [ OK ] Started Mark the need to relabel after reboot. [ OK ] Started Load/Save Random Seed. [ OK ] Started Create Volatile Files and Directories. Starting RPC Bind... Starting Update UTMP about System Boot/Shutdown... [ OK ] Started Update UTMP about System Boot/Shutdown. [ OK ] Started RPC Bind. [ OK ] Started Rebuild Dynamic Linker Cache. Starting Update is Completed... [ OK ] Started Update is Completed. [ OK ] Reached target System Initialization. [ OK ] Started dnf makecache --timer. [ OK ] Listening on D-Bus System Message Bus Socket. [ OK ] Reached target Sockets. [ OK ] Reached target Basic System. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. Starting Login Service... [ OK ] Started irqbalance daemon. Starting Restore /run/initramfs on shutdown... [ OK ] Started D-Bus System Message Bus. Starting Network Manager... [ OK ] Started Daily Cleanup of Temporary Directories. [ OK ] Reached target sshd-keygen.target. [ OK ] Started daily update of the root trust anchor for DNSSEC. [ OK ] Reached target Timers. [ OK ] Started Restore /run/initramfs on shutdown. [ OK ] Started Login Service. [ OK ] Started Network Manager. [ OK ] Reached target Network. Starting OpenSSH server daemon... Starting Dynamic System Tuning Daemon... Starting GSSAPI Proxy Daemon... Starting Network Manager Wait Online... [ OK ] Started OpenSSH server daemon. Starting Hostname Service... [ OK ] Started GSSAPI Proxy Daemon. [ OK ] Reached target NFS client services. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Starting Permit User Sessions... [ OK ] Started Permit User Sessions. [ OK ] Started Getty on tty1. [ OK ] Started Serial Getty on ttyS0. [ OK ] Started Serial Getty on ttyS1. [ OK ] Reached target Login Prompts. [ OK ] Started Command Scheduler. [ OK ] Started Hostname Service. Starting Network Manager Script Dispatcher Service... [ OK ] Started Network Manager Script Dispatcher Service. [ OK ] Started Network Manager Wait Online. [ OK ] Reached target Network is Online. Starting Crash recovery kernel arming... Starting Notify NFS peers of a restart... Starting System Logging Service... [ OK ] Started Notify NFS peers of a restart. [ OK ] Started System Logging Service. Rocky Linux 8.10 (Green Obsidian) Kernel 4.18.0rh8.10-debug on an x86_64 oleg358-server login: [ 88.591573] libcfs: loading out-of-tree module taints kernel. [ 88.638459] Key type ._llcrypt registered [ 88.640752] Key type .llcrypt registered [ 88.740887] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing set_hostid [ 104.554709] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing load_modules_local [ 106.042270] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 106.060647] alg: No test for adler32 (adler32-zlib) [ 107.619695] Lustre: Lustre: Build Version: 2.17.52_93_g6cdb58a [ 108.679468] LNet: Added LNI 192.168.203.158@tcp [8/256/0/180] [ 110.481364] Key type lgssc registered [ 112.244866] Lustre: Echo OBD driver; http://www.lustre.org/ [ 112.388424] hrtimer: interrupt took 2743974 ns [ 131.349064] ZFS: Loaded module v2.3.2-1, ZFS pool version 5000, ZFS filesystem version 5 [ 176.986346] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing load_modules_local [ 192.691421] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 192.727269] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 194.089625] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 194.138399] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 194.275388] Lustre: lustre-MDT0000: new disk, initializing [ 194.367456] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 194.388701] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 198.964403] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 211.496797] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 211.590910] Lustre: 6504:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 211.612243] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 211.615697] Lustre: Skipped 1 previous similar message [ 211.694867] Lustre: lustre-MDT0001: new disk, initializing [ 211.751144] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 211.786918] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 211.802146] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 216.176677] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 221.709479] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 231.725163] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 232.051456] Lustre: lustre-OST0000: new disk, initializing [ 232.057253] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 232.138290] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 235.538058] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 235.546566] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 235.673155] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 239.828361] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 255.034873] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 255.221729] Lustre: lustre-OST0001: new disk, initializing [ 255.233782] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 255.324588] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 261.654372] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 261.673457] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 261.761370] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 262.103496] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 273.408564] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 279.242668] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 285.912352] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing check_logdir /tmp/testlogs/ [ 290.972397] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing yml_node [ 296.457105] Lustre: DEBUG MARKER: Client: 2.17.52.93 [ 299.069320] Lustre: DEBUG MARKER: MDS: 2.17.52.93 [ 301.433994] Lustre: DEBUG MARKER: OSS: 2.17.52.93 [ 302.784310] Lustre: DEBUG MARKER: -----============= acceptance-small: conf-sanity ============----- Wed Apr 29 16:22:53 EDT 2026 [ 318.397184] Lustre: DEBUG MARKER: excepting tests: 102 106 115 32newtarball [ 319.667904] Lustre: DEBUG MARKER: skipping tests SLOW=no: 45 69 106 111 114 [ 338.400511] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 338.401265] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 338.428628] Lustre: Skipped 3 previous similar messages [ 338.448878] Lustre: Skipped 3 previous similar messages [ 343.213351] Lustre: server umount lustre-MDT0000 complete [ 346.593845] LustreError: 6494:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1777494219 with bad export cookie 235247559252681012 [ 346.597627] LustreError: MGC192.168.203.158@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 346.601189] LustreError: 6494:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 348.641693] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 348.642222] LustreError: 6510:0:(ldlm_lib.c:1180:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 348.642614] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 348.655897] Lustre: Skipped 1 previous similar message [ 348.687316] LustreError: 6510:0:(ldlm_lib.c:1180:target_handle_connect()) Skipped 6 previous similar messages [ 351.647963] LustreError: 8416:0:(ldlm_lib.c:1180:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 351.649864] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 351.675669] LustreError: 8416:0:(ldlm_lib.c:1180:target_handle_connect()) Skipped 1 previous similar message [ 351.680993] Lustre: Skipped 2 previous similar messages [ 352.958780] Lustre: server umount lustre-MDT0001 complete [ 356.912322] Lustre: server umount lustre-OST0000 complete [ 360.992118] Lustre: server umount lustre-OST0001 complete [ 367.529854] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing set_hostid [ 377.025624] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing load_modules_local [ 415.310964] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 415.731407] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 415.782469] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 415.896209] Lustre: lustre-MDT0000: new disk, initializing [ 416.063787] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 416.096583] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 421.518426] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing set_default_debug -1 all [ 431.057287] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 431.182577] Lustre: 14979:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 431.233750] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 431.237679] Lustre: Skipped 1 previous similar message [ 431.299379] Lustre: lustre-MDT0001: new disk, initializing [ 431.378393] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 431.408311] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 431.423216] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 435.820988] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing set_default_debug -1 all [ 444.109710] Lustre: DEBUG MARKER: oleg358-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 447.665986] Lustre: DEBUG MARKER: oleg358-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 454.430563] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 454.726433] Lustre: lustre-OST0000: new disk, initializing [ 454.736449] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 454.860983] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 456.146038] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 456.163393] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 456.243611] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 461.130082] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing set_default_debug -1 all [ 468.369167] Lustre: DEBUG MARKER: oleg358-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 472.866889] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 473.078473] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 476.841311] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 477.080446] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 481.770930] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 481.773308] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 481.788793] LustreError: Skipped 1 previous similar message [ 481.797906] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 481.810102] Lustre: Skipped 1 previous similar message [ 484.853158] Lustre: server umount lustre-OST0000 complete [ 492.000100] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 492.007105] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 492.021869] Lustre: Skipped 2 previous similar messages [ 494.334831] Lustre: server umount lustre-MDT0000 complete [ 497.126975] LustreError: 16363:0:(ldlm_lib.c:1180:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 497.162071] LustreError: 16363:0:(ldlm_lib.c:1180:target_handle_connect()) Skipped 1 previous similar message [ 498.145316] LustreError: 14970:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1777494371 with bad export cookie 235247559252682545 [ 498.149984] LustreError: MGC192.168.203.158@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 498.157832] LustreError: 14970:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 498.537148] Lustre: server umount lustre-MDT0001 complete [ 503.226867] Lustre: DEBUG MARKER: == conf-sanity test 76a: set permanent params with lctl across mounts ========================================================== 16:26:14 (1777494374) [ 509.327729] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 509.801555] LustreError: 18839:0:(ldlm_lib.c:1180:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 509.872037] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 514.163777] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing set_default_debug -1 all [ 515.049479] LustreError: 18840:0:(ldlm_lib.c:1180:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 523.041440] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 527.375924] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing set_default_debug -1 all [ 533.041718] Lustre: DEBUG MARKER: oleg358-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 536.996556] Lustre: DEBUG MARKER: oleg358-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 543.234647] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 549.349808] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing set_default_debug -1 all [ 554.404797] Lustre: DEBUG MARKER: oleg358-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 563.042127] Lustre: Modifying parameter general.osc.*.max_dirty_mb=452 in log params [ 569.316598] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 569.325986] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 569.355461] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 569.360079] Lustre: Skipped 1 previous similar message [ 574.607124] Lustre: server umount lustre-MDT0000 complete [ 575.457404] LustreError: 18839:0:(ldlm_lib.c:1180:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 575.501505] LustreError: 18839:0:(ldlm_lib.c:1180:target_handle_connect()) Skipped 3 previous similar messages [ 578.543405] LustreError: 18821:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1777494451 with bad export cookie 235247559252683427 [ 578.547468] LustreError: MGC192.168.203.158@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 578.564088] LustreError: 18821:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 579.076974] Lustre: server umount lustre-MDT0001 complete [ 593.972083] Lustre: server umount lustre-OST0000 complete [ 615.815494] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing load_modules_local [ 625.642205] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 626.074775] LustreError: 22866:0:(ldlm_lib.c:1180:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 626.171033] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 626.179570] Lustre: Skipped 2 previous similar messages [ 629.960660] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing set_default_debug -1 all [ 637.992018] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 642.292880] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing set_default_debug -1 all [ 644.951132] Lustre: Setting parameter general.debug_raw_pointers=Y in log params [ 651.559855] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 657.184751] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing set_default_debug -1 all [ 664.320801] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 664.500063] Lustre: lustre-OST0001: new disk, initializing [ 664.504762] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 665.725487] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 665.735838] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 665.851629] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 670.584739] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing set_default_debug -1 all [ 680.918806] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 684.254524] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 696.804141] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 696.807687] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 696.816993] Lustre: Skipped 5 previous similar messages [ 696.838034] Lustre: Skipped 6 previous similar messages [ 702.333982] Lustre: server umount lustre-MDT0000 complete [ 706.074427] LustreError: 23589:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1777494578 with bad export cookie 235247559252684274 [ 706.085156] LustreError: MGC192.168.203.158@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 706.085992] LustreError: 23589:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 706.500803] Lustre: server umount lustre-MDT0001 complete [ 721.071700] Lustre: server umount lustre-OST0000 complete [ 722.386514] Lustre: 3655:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1777494579/real 1777494579] req@ffff952c41eb0a80 x1863837528858752/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1777494595 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 722.415338] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 725.038455] Lustre: server umount lustre-OST0001 complete [ 741.318960] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing load_modules_local [ 752.123952] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 752.815781] LustreError: 28117:0:(ldlm_lib.c:1180:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 752.853651] LustreError: 28117:0:(ldlm_lib.c:1180:target_handle_connect()) Skipped 3 previous similar messages [ 757.446483] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing set_default_debug -1 all [ 767.465362] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 767.761363] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 767.767724] Lustre: Skipped 4 previous similar messages [ 771.881559] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing set_default_debug -1 all [ 774.805956] Lustre: 29232:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 781.577437] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 788.564308] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing set_default_debug -1 all [ 797.229747] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 804.204538] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing set_default_debug -1 all [ 811.864252] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 815.232297] Lustre: 31076:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 824.290909] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 824.304923] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 824.326348] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 824.336822] Lustre: Skipped 7 previous similar messages [ 826.816282] Lustre: server umount lustre-MDT0000 complete [ 829.409946] LustreError: 28117:0:(ldlm_lib.c:1180:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 829.428660] LustreError: 28117:0:(ldlm_lib.c:1180:target_handle_connect()) Skipped 7 previous similar messages [ 830.187905] LustreError: 28098:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1777494703 with bad export cookie 235247559252685569 [ 830.190643] LustreError: MGC192.168.203.158@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 830.200082] LustreError: 28098:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 830.479460] Lustre: server umount lustre-MDT0001 complete [ 844.893314] Lustre: server umount lustre-OST0000 complete [ 859.224208] Lustre: server umount lustre-OST0001 complete [ 869.263157] Lustre: DEBUG MARKER: == conf-sanity test 76b: verify params log setup correctly ========================================================== 16:32:19 (1777494739) [ 898.176205] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing load_modules_local [ 908.649713] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 913.918434] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing set_default_debug -1 all [ 923.114901] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 928.505318] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing set_default_debug -1 all [ 931.356563] Lustre: 34993:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 938.527766] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 946.868994] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing set_default_debug -1 all [ 958.295544] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 966.856157] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing set_default_debug -1 all [ 973.726789] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 977.613864] Lustre: 36838:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 984.549813] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 984.551073] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 984.567531] Lustre: Skipped 5 previous similar messages [ 984.583735] Lustre: Skipped 3 previous similar messages [ 988.727886] Lustre: server umount lustre-MDT0000 complete [ 989.683832] LustreError: 33877:0:(ldlm_lib.c:1180:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 989.725256] LustreError: 33877:0:(ldlm_lib.c:1180:target_handle_connect()) Skipped 10 previous similar messages [ 992.614867] LustreError: 33859:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1777494865 with bad export cookie 235247559252686675 [ 992.623287] LustreError: MGC192.168.203.158@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 992.632685] LustreError: 33859:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 993.031610] Lustre: server umount lustre-MDT0001 complete [ 1007.677527] Lustre: server umount lustre-OST0000 complete [ 1010.655922] Lustre: 3653:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1777494867/real 1777494867] req@ffff952d5e08ed80 x1863837528956544/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1777494883 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1011.966917] Lustre: server umount lustre-OST0001 complete [ 1021.323916] Lustre: DEBUG MARKER: == conf-sanity test 76c: verify changelog_mask is applied with lctl set_param -P ========================================================== 16:34:52 (1777494892) [ 1036.896076] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing load_modules_local [ 1048.092801] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1048.767455] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1048.774609] Lustre: Skipped 6 previous similar messages [ 1053.167543] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing set_default_debug -1 all [ 1062.688842] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1067.326438] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing set_default_debug -1 all [ 1070.319513] Lustre: 40077:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1077.214260] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1083.466577] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing set_default_debug -1 all [ 1093.666294] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1100.070956] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing set_default_debug -1 all [ 1108.110555] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1111.599869] Lustre: 41921:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 1112.658671] Lustre: Setting parameter general.mdd.*.changelog_mask=-CLOSE in log params [ 1112.669040] Lustre: Skipped 1 previous similar message [ 1115.621335] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1115.639094] Lustre: Skipped 2 previous similar messages [ 1115.643908] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1115.659681] Lustre: Skipped 3 previous similar messages [ 1121.898777] Lustre: server umount lustre-MDT0000 complete [ 1125.554405] LustreError: 38945:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1777494998 with bad export cookie 235247559252687781 [ 1125.563968] LustreError: MGC192.168.203.158@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1125.580610] LustreError: 38945:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 1132.122953] Lustre: server umount lustre-MDT0001 complete [ 1141.034209] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1156.616400] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing set_default_debug -1 all [ 1161.213962] Lustre: lustre-MDT0000-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 1165.520961] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1170.887602] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing set_default_debug -1 all [ 1171.437656] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 1171.450016] Lustre: Skipped 1 previous similar message [ 1177.567741] Lustre: DEBUG MARKER: oleg358-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1179.714347] Lustre: DEBUG MARKER: mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec [ 1185.044238] Lustre: DEBUG MARKER: oleg358-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 1187.047176] Lustre: DEBUG MARKER: mdc.lustre-MDT0001-mdc-*.mds_server_uuid in FULL state after 0 sec [ 1197.040024] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1199.216930] Lustre: server umount lustre-MDT0000 complete [ 1203.205965] LustreError: 38943:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1777495076 with bad export cookie 235247559252688908 [ 1203.209903] LustreError: MGC192.168.203.158@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1203.226928] LustreError: 38943:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1203.930324] Lustre: server umount lustre-MDT0001 complete [ 1219.694810] Lustre: server umount lustre-OST0000 complete [ 1223.648707] Lustre: 3653:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1777495080/real 1777495080] req@ffff952d47ab8000 x1863837529041920/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1777495096 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1224.289412] Lustre: server umount lustre-OST0001 complete [ 1233.782686] Lustre: DEBUG MARKER: == conf-sanity test 76d: verify llite.*.xattr_cache can be set by 'lctl set_param -P' correctly ========================================================== 16:38:24 (1777495104) [ 1250.254971] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing load_modules_local [ 1261.744448] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1262.294663] LustreError: 46287:0:(ldlm_lib.c:1180:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1262.326413] LustreError: 46287:0:(ldlm_lib.c:1180:target_handle_connect()) Skipped 22 previous similar messages [ 1267.941996] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing set_default_debug -1 all [ 1278.063290] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1283.176628] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing set_default_debug -1 all [ 1286.564946] Lustre: 47407:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1293.138858] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1299.885876] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing set_default_debug -1 all [ 1310.121700] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1316.984930] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing set_default_debug -1 all [ 1324.657410] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1328.777610] Lustre: Setting parameter general.llite.*.xattr_cache=0 in log params [ 1347.608936] Lustre: server umount lustre-MDT0000 complete [ 1351.309753] LustreError: 46267:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1777495224 with bad export cookie 235247559252689699 [ 1351.311989] LustreError: MGC192.168.203.158@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1351.322068] LustreError: 46267:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 1351.792794] Lustre: server umount lustre-MDT0001 complete [ 1358.526780] Lustre: server umount lustre-OST0000 complete [ 1362.908885] Lustre: server umount lustre-OST0001 complete [ 1373.223809] Lustre: DEBUG MARKER: == conf-sanity test 77: comma-separated MGS NIDs and failover node NIDs ========================================================== 16:40:44 (1777495244) [ 1376.608819] Lustre: DEBUG MARKER: SKIP: conf-sanity test_77 mixed loopback and real device not working [ 1378.903171] Lustre: DEBUG MARKER: == conf-sanity test 78: run resize2fs on MDT and OST filesystems ========================================================== 16:40:49 (1777495249) [ 1393.721507] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing set_hostid [ 1404.753962] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing load_modules_local [ 1431.473098] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1431.851405] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1431.959955] Lustre: lustre-MDT0000: new disk, initializing [ 1432.078327] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1437.447833] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing set_default_debug -1 all [ 1445.208551] Lustre: DEBUG MARKER: oleg358-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1452.019269] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1452.325996] Lustre: lustre-OST0000: new disk, initializing [ 1452.339369] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1452.357159] Lustre: Skipped 1 previous similar message [ 1453.744080] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 1453.756609] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 1453.801635] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 1458.861983] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing set_default_debug -1 all [ 1466.097177] Lustre: DEBUG MARKER: oleg358-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1468.515616] Lustre: DEBUG MARKER: create test files [ 1500.128796] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1500.150388] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1500.167304] Lustre: Skipped 14 previous similar messages [ 1500.196457] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1500.211237] Lustre: Skipped 22 previous similar messages [ 1503.306199] Lustre: server umount lustre-OST0000 complete [ 1510.029859] Lustre: server umount lustre-MDT0000 complete [ 1523.534718] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing unload_modules_local [ 1526.893969] Key type lgssc unregistered [ 1527.174438] LNet: 54798:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1527.195076] LNetError: 54798:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1528.232372] LNet: Removed LNI 192.168.203.158@tcp [ 1529.411283] Key type .llcrypt unregistered [ 1529.412916] Key type ._llcrypt unregistered [ 1557.697428] Key type ._llcrypt registered [ 1557.701467] Key type .llcrypt registered [ 1557.823848] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing load_modules_local [ 1558.957307] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1558.988350] alg: No test for adler32 (adler32-zlib) [ 1560.079730] Lustre: Lustre: Build Version: 2.17.52_93_g6cdb58a [ 1560.311053] LNet: Added LNI 192.168.203.158@tcp [8/256/0/180] [ 1562.007342] Key type lgssc registered [ 1563.151293] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1570.547643] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1570.579224] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1572.161479] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1577.256622] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing set_default_debug -1 all [ 1583.133982] Lustre: DEBUG MARKER: oleg358-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1590.421441] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1590.754282] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1597.112082] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing set_default_debug -1 all [ 1597.958505] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:102 to 0x240000400:129) [ 1598.086863] Lustre: 57404:0:(osd_internal.h:1447:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 515 < left 544, rollback = 9 [ 1598.099918] Lustre: 57404:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 32/128/0, destroy: 0/0/0 [ 1598.114855] Lustre: 57404:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 1598.122914] Lustre: 57404:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/18/1, punch: 0/0/0, quota 1/3/0 [ 1598.144285] Lustre: 57404:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 1598.172787] Lustre: 57404:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 1603.237307] Lustre: DEBUG MARKER: oleg358-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1608.252458] Lustre: DEBUG MARKER: check files after expanding the MDT and OST filesystems [ 1613.234443] Lustre: DEBUG MARKER: create more files after expanding the MDT and OST filesystems [ 1621.492984] Lustre: server umount lustre-OST0000 complete [ 1634.474878] Lustre: server umount lustre-MDT0000 complete [ 1644.668392] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing unload_modules_local [ 1647.353688] Key type lgssc unregistered [ 1647.671583] LNet: 58885:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1647.681625] LNetError: 58885:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1647.702879] LNet: Removed LNI 192.168.203.158@tcp [ 1648.579588] Key type .llcrypt unregistered [ 1648.581911] Key type ._llcrypt unregistered [ 1678.297888] Key type ._llcrypt registered [ 1678.308171] Key type .llcrypt registered [ 1678.463662] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing load_modules_local [ 1679.759447] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1679.793700] alg: No test for adler32 (adler32-zlib) [ 1680.873435] Lustre: Lustre: Build Version: 2.17.52_93_g6cdb58a [ 1681.191624] LNet: Added LNI 192.168.203.158@tcp [8/256/0/180] [ 1682.919172] Key type lgssc registered [ 1684.514730] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1693.620347] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1693.671548] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1695.629789] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1701.505887] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing set_default_debug -1 all [ 1707.194853] Lustre: DEBUG MARKER: oleg358-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1713.999901] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1714.372371] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1720.500799] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:140 to 0x240000400:161) [ 1720.567630] Lustre: 61587:0:(osd_internal.h:1447:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 509 < left 544, rollback = 9 [ 1720.574708] Lustre: 61587:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 32/128/6, destroy: 0/0/0 [ 1720.591281] Lustre: 61587:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 1720.604209] Lustre: 61587:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/18/1, punch: 0/0/0, quota 1/3/0 [ 1720.617565] Lustre: 61587:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 1720.627459] Lustre: 61587:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 1721.135835] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing set_default_debug -1 all [ 1727.183810] Lustre: DEBUG MARKER: oleg358-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1732.097410] Lustre: DEBUG MARKER: check files after shrinking the MDT and OST filesystems [ 1741.287858] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1741.296276] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1741.300436] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1746.408961] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1747.181766] Lustre: server umount lustre-OST0000 complete [ 1751.612106] Lustre: server umount lustre-MDT0000 complete [ 1762.028400] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing unload_modules_local [ 1764.840455] Key type lgssc unregistered [ 1765.211283] LNet: 63014:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1765.225694] LNetError: 63014:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1765.254467] LNet: Removed LNI 192.168.203.158@tcp [ 1766.052264] Key type .llcrypt unregistered [ 1766.055283] Key type ._llcrypt unregistered [ 1789.016496] Key type ._llcrypt registered [ 1789.022441] Key type .llcrypt registered [ 1789.202693] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing set_hostid [ 1805.186507] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing load_modules_local [ 1806.391092] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1806.464056] alg: No test for adler32 (adler32-zlib) [ 1807.477284] Lustre: Lustre: Build Version: 2.17.52_93_g6cdb58a [ 1807.727332] LNet: Added LNI 192.168.203.158@tcp [8/256/0/180] [ 1809.351162] Key type lgssc registered [ 1810.569644] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1847.331948] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1847.378346] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1848.627573] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 1848.689775] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1848.812337] Lustre: lustre-MDT0000: new disk, initializing [ 1848.940229] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1848.976610] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1852.952170] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing set_default_debug -1 all [ 1863.790851] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1863.946543] Lustre: 66132:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 1864.011950] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 1864.022813] Lustre: Skipped 1 previous similar message [ 1864.154501] Lustre: lustre-MDT0001: new disk, initializing [ 1864.300053] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 1864.333573] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 1864.352421] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 1868.541890] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing set_default_debug -1 all [ 1875.847469] Lustre: DEBUG MARKER: oleg358-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1879.247499] Lustre: DEBUG MARKER: oleg358-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 1885.647889] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1885.953631] Lustre: lustre-OST0000: new disk, initializing [ 1885.968620] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1886.069333] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1892.012698] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing set_default_debug -1 all [ 1894.463182] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 1894.471719] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 1894.598963] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 1899.337518] Lustre: DEBUG MARKER: oleg358-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1904.970437] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 1905.364345] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 1911.324482] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 1911.725225] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 1915.360972] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1915.363969] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1915.384832] LustreError: Skipped 1 previous similar message [ 1915.388311] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1915.409564] Lustre: Skipped 1 previous similar message [ 1915.443273] Lustre: Skipped 1 previous similar message [ 1920.753652] Lustre: server umount lustre-OST0000 complete [ 1930.721144] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1930.725630] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1930.729913] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1930.761988] Lustre: Skipped 1 previous similar message [ 1932.957750] Lustre: server umount lustre-MDT0000 complete [ 1935.840121] LustreError: 66144:0:(ldlm_lib.c:1180:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1935.850491] LustreError: 66144:0:(ldlm_lib.c:1180:target_handle_connect()) Skipped 4 previous similar messages [ 1937.308465] LustreError: 66126:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1777495810 with bad export cookie 2244971675758577667 [ 1937.318653] LustreError: MGC192.168.203.158@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1937.337393] LustreError: 66126:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1937.713808] Lustre: server umount lustre-MDT0001 complete [ 1947.266657] Lustre: DEBUG MARKER: == conf-sanity test 79: format MDT/OST without mgs option (should return errors) ========================================================== 16:50:17 (1777495817) [ 1960.552877] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing load_modules_local [ 1997.162584] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing set_hostid [ 2007.832809] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing load_modules_local [ 2044.565465] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2044.944659] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2044.991669] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2045.107805] Lustre: lustre-MDT0000: new disk, initializing [ 2045.233951] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2045.266366] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2049.792939] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing set_default_debug -1 all [ 2060.615502] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2060.749915] Lustre: 73696:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2060.815318] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 2060.819191] Lustre: Skipped 1 previous similar message [ 2060.911738] Lustre: lustre-MDT0001: new disk, initializing [ 2061.033943] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2061.074765] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 2061.090635] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2065.704556] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing set_default_debug -1 all [ 2073.833585] Lustre: DEBUG MARKER: oleg358-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2077.335232] Lustre: DEBUG MARKER: oleg358-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2083.842056] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2084.172376] Lustre: lustre-OST0000: new disk, initializing [ 2084.179850] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 2084.295673] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2091.399940] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing set_default_debug -1 all [ 2091.625631] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 2091.645550] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2091.855920] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2100.362903] Lustre: DEBUG MARKER: oleg358-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2105.224373] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2105.438889] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2110.209207] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2110.474734] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2117.615070] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2117.621616] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2117.626468] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2117.626474] Lustre: Skipped 1 previous similar message [ 2117.664366] Lustre: Skipped 1 previous similar message [ 2118.895466] Lustre: server umount lustre-OST0000 complete [ 2122.721901] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2122.731598] LustreError: Skipped 1 previous similar message [ 2122.736680] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2122.771550] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2122.778668] Lustre: Skipped 2 previous similar messages [ 2127.841315] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2127.850853] Lustre: Skipped 1 previous similar message [ 2128.887822] Lustre: server umount lustre-MDT0000 complete [ 2132.968161] LustreError: 73708:0:(ldlm_lib.c:1180:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2132.982206] LustreError: 73708:0:(ldlm_lib.c:1180:target_handle_connect()) Skipped 1 previous similar message [ 2133.172473] LustreError: 73688:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1777496006 with bad export cookie 2244971675758578549 [ 2133.196941] LustreError: MGC192.168.203.158@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2133.526497] Lustre: server umount lustre-MDT0001 complete [ 2145.148718] Lustre: DEBUG MARKER: == conf-sanity test 80: mgc import reconnect race ======== 16:53:35 (1777496015) [ 2153.055200] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2153.625367] LustreError: 77703:0:(ldlm_lib.c:1180:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2153.745692] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2159.073559] LustreError: 77704:0:(ldlm_lib.c:1180:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2159.512778] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing set_default_debug -1 all [ 2163.169337] LustreError: 77703:0:(ldlm_lib.c:1180:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2168.442581] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2175.220685] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing set_default_debug -1 all [ 2182.158175] Lustre: DEBUG MARKER: oleg358-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2186.332518] Lustre: DEBUG MARKER: oleg358-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2193.216338] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2193.377571] LustreError: 79295:0:(ldlm_lib.c:1180:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2193.407736] LustreError: 79295:0:(ldlm_lib.c:1180:target_handle_connect()) Skipped 1 previous similar message [ 2193.596351] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2193.613736] Lustre: Skipped 1 previous similar message [ 2200.883896] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing set_default_debug -1 all [ 2206.731767] Lustre: DEBUG MARKER: oleg358-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2210.463894] Lustre: 80037:0:(genops.c:1795:obd_export_evict_by_uuid()) MGS: evicting 63e80177-cefb-4a0f-850b-64f07f6bb89f at adminstrative request [ 2214.371814] LustreError: MGC192.168.203.158@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2214.394644] Lustre: *** cfs_fail_loc=906, val=2147483648*** [ 2224.610407] Lustre: MGS: Client 63e80177-cefb-4a0f-850b-64f07f6bb89f (at 0@lo) reconnecting [ 2224.625594] Lustre: Evicted from MGS (at 0@lo) after server handle changed from 0x1f27bdd840674ee7 to 0x1f27bdd840675187 [ 2224.636469] Lustre: MGC192.168.203.158@tcp: Connection restored to 0@lo (at 0@lo) [ 2248.106835] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2248.307926] Lustre: lustre-OST0001: new disk, initializing [ 2248.312060] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 2254.431332] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing set_default_debug -1 all [ 2254.969991] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 2254.982148] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 2255.071875] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 2262.307024] Lustre: DEBUG MARKER: oleg358-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid [ 2265.574981] Lustre: lustre-OST0001-osc-MDT0000: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2265.578362] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 2265.591977] Lustre: Skipped 2 previous similar messages [ 2265.611354] Lustre: Skipped 1 previous similar message [ 2271.827704] Lustre: server umount lustre-OST0001 complete [ 2275.831220] LustreError: 79298:0:(ldlm_lib.c:1180:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2275.858196] LustreError: 79298:0:(ldlm_lib.c:1180:target_handle_connect()) Skipped 1 previous similar message [ 2280.935912] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2280.936301] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2280.952550] Lustre: Skipped 1 previous similar message [ 2280.984897] LustreError: Skipped 1 previous similar message [ 2282.458652] Lustre: server umount lustre-OST0000 complete [ 2286.561409] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2286.568262] Lustre: Skipped 4 previous similar messages [ 2292.911490] Lustre: server umount lustre-MDT0000 complete [ 2296.809771] LustreError: 77685:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1777496169 with bad export cookie 2244971675758580103 [ 2296.822435] LustreError: MGC192.168.203.158@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2296.824034] LustreError: 77685:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 8 previous similar messages [ 2297.387919] Lustre: server umount lustre-MDT0001 complete [ 2307.364811] Lustre: DEBUG MARKER: == conf-sanity test 81: sparse OST indexing ============== 16:56:17 (1777496177) [ 2309.206177] Lustre: DEBUG MARKER: SKIP: conf-sanity test_81 needs >= 3 OSTs [ 2311.550739] Lustre: DEBUG MARKER: == conf-sanity test 82a: specify OSTs for file (succeed) or directory (succeed) ========================================================== 16:56:21 (1777496181) [ 2313.742045] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82a needs >= 3 OSTs [ 2316.234125] Lustre: DEBUG MARKER: == conf-sanity test 82b: specify OSTs for file with --pool and --ost-list options ========================================================== 16:56:26 (1777496186) [ 2318.477093] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82b needs >= 4 OSTs [ 2320.475842] Lustre: DEBUG MARKER: == conf-sanity test 82c: specify sparse OSTs for setstripe (should not crash) ========================================================== 16:56:31 (1777496191) [ 2322.323055] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82c needs >= 4 OSTs [ 2324.417993] Lustre: DEBUG MARKER: == conf-sanity test 83: ENOSPACE on OST doesn't cause message VFS: Busy inodes after unmount ... ========================================================== 16:56:34 (1777496194) [ 2326.370726] Lustre: DEBUG MARKER: mount the OST /dev/mapper/ost1_flakey as a ldiskfs filesystem [ 2335.165055] loop: module loaded [ 2335.193867] loop0: detected capacity change from 0 to 4294967296 [ 2335.269844] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 2337.447991] Lustre: DEBUG MARKER: run llverfs in partial mode on the OST ldiskfs /mnt/lustre-ost1 [ 2341.608081] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing run_llverfs /mnt/lustre-ost1 -vpl no [ 2345.498720] Lustre: DEBUG MARKER: unmount the OST /dev/mapper/ost1_flakey [ 2374.570653] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2374.638790] LustreError: 84428:0:(obd_config.c:845:class_setup()) setup lustre-OST0000-osd failed (-28) [ 2374.659088] LustreError: 84428:0:(obd_mount.c:250:lustre_start_simple()) lustre-OST0000-osd setup error -28 [ 2374.674500] LustreError: 84428:0:(tgt_mount.c:2534:server_fill_super()) Unable to start osd on /dev/mapper/ost1_flakey: -28 [ 2374.690747] LustreError: 84428:0:(super25.c:184:lustre_fill_super()) llite: Unable to mount : rc = -28 [ 2396.536981] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing set_hostid [ 2408.060396] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing load_modules_local [ 2447.499352] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2447.839795] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2447.877397] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2447.983988] Lustre: lustre-MDT0000: new disk, initializing [ 2448.133030] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2448.136379] Lustre: Skipped 1 previous similar message [ 2448.153810] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2453.168178] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing set_default_debug -1 all [ 2464.512729] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2464.658205] Lustre: 87259:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2465.052934] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2470.315053] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing set_default_debug -1 all [ 2478.313306] Lustre: DEBUG MARKER: oleg358-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2483.390449] Lustre: DEBUG MARKER: oleg358-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2491.504624] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2499.336705] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing set_default_debug -1 all [ 2502.230956] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2502.343022] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2507.452843] Lustre: DEBUG MARKER: oleg358-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2511.834690] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2512.038818] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2517.579610] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2517.876402] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2523.108918] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2523.113633] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2523.134774] LustreError: Skipped 1 previous similar message [ 2523.161595] Lustre: Skipped 2 previous similar messages [ 2523.176839] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2523.187797] Lustre: Skipped 3 previous similar messages [ 2526.503710] Lustre: server umount lustre-OST0000 complete [ 2537.121887] Lustre: server umount lustre-MDT0000 complete [ 2538.464426] LustreError: 88519:0:(ldlm_lib.c:1180:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2538.483940] LustreError: 88519:0:(ldlm_lib.c:1180:target_handle_connect()) Skipped 5 previous similar messages [ 2542.099778] LustreError: 87254:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1777496415 with bad export cookie 2244971675758580719 [ 2542.108647] LustreError: MGC192.168.203.158@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2542.119432] LustreError: 87254:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2542.555983] Lustre: server umount lustre-MDT0001 complete [ 2551.986446] Lustre: DEBUG MARKER: == conf-sanity test 84: check recovery_hard_time ========= 17:00:22 (1777496422) [ 2559.271719] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2564.864658] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing set_default_debug -1 all [ 2573.902390] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2579.089291] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing set_default_debug -1 all [ 2585.177579] Lustre: DEBUG MARKER: oleg358-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2589.318838] Lustre: DEBUG MARKER: oleg358-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2598.171422] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2598.696111] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2598.704950] Lustre: Skipped 4 previous similar messages [ 2604.841796] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing set_default_debug -1 all [ 2611.556605] Lustre: DEBUG MARKER: oleg358-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2617.538921] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2617.803863] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2623.306733] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2623.506141] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2631.511184] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2631.770393] Lustre: lustre-OST0001: new disk, initializing [ 2631.775690] Lustre: Skipped 2 previous similar messages [ 2631.782816] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 2631.790220] Lustre: Skipped 3 previous similar messages [ 2639.543717] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing set_default_debug -1 all [ 2641.604590] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 2641.619519] Lustre: Skipped 2 previous similar messages [ 2641.632172] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 2641.669507] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 2648.458276] Lustre: DEBUG MARKER: oleg358-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid [ 2654.253231] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid 50 [ 2654.629274] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2660.534562] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [ 2660.894188] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2681.595886] Lustre: DEBUG MARKER: mds1 REPLAY BARRIER on lustre-MDT0000 [ 2713.593707] Lustre: Failing over lustre-MDT0000 [ 2714.041803] Lustre: server umount lustre-MDT0000 complete [ 2714.532146] LustreError: 91265:0:(ldlm_lib.c:1180:target_handle_connect()) lustre-MDT0000: not available for connect from 192.168.203.58@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2714.546159] LustreError: 91265:0:(ldlm_lib.c:1180:target_handle_connect()) Skipped 4 previous similar messages [ 2717.666695] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2717.677503] LustreError: Skipped 2 previous similar messages [ 2717.687098] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2717.704319] Lustre: Skipped 3 previous similar messages [ 2735.073675] Lustre: 64359:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1777496591/real 1777496591] req@000000002b48a189 x1863839310425728/t0(0) o400->MGC192.168.203.158@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1777496607 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 2735.113590] LustreError: MGC192.168.203.158@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2741.990511] LDISKFS-fs (dm-0): 4 truncates cleaned up [ 2741.992654] LDISKFS-fs (dm-0): recovery complete [ 2742.019665] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2745.780989] Lustre: lustre-MDT0000: in recovery but waiting for the first client to connect [ 2750.256584] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing set_default_debug -1 all [ 2750.364989] Lustre: lustre-MDT0000: Will be in recovery for at least 1:00, or until 3 clients reconnect [ 2750.372405] Lustre: 91266:0:(ldlm_lib.c:2070:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 2750.963326] Lustre: 91981:0:(ldlm_lib.c:2070:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 2750.985140] Lustre: 91981:0:(ldlm_lib.c:2070:extend_recovery_timer()) Skipped 1 previous similar message [ 2751.015180] Lustre: lustre-MDT0000-lwp-OST0001: Connection restored to 0@lo (at 0@lo) [ 2751.105153] LustreError: 96992:0:(ldlm_lib.c:2683:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2751.438534] LustreError: 96992:0:(ldlm_lib.c:2683:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2751.791219] LustreError: 96992:0:(ldlm_lib.c:2683:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2751.802680] LustreError: 96992:0:(ldlm_lib.c:2683:replay_request_or_update()) Skipped 1 previous similar message [ 2752.136131] LustreError: 96992:0:(ldlm_lib.c:2683:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2752.138856] LustreError: 96992:0:(ldlm_lib.c:2683:replay_request_or_update()) Skipped 1 previous similar message [ 2752.159116] Lustre: 96992:0:(ldlm_lib.c:2070:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2752.174175] Lustre: 96992:0:(ldlm_lib.c:2070:extend_recovery_timer()) Skipped 7 previous similar messages [ 2752.825088] LustreError: 96992:0:(ldlm_lib.c:2683:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2752.840835] LustreError: 96992:0:(ldlm_lib.c:2683:replay_request_or_update()) Skipped 2 previous similar messages [ 2753.175073] LustreError: 96992:0:(ldlm_lib.c:2683:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2753.192345] LustreError: 96992:0:(ldlm_lib.c:2683:replay_request_or_update()) Skipped 2 previous similar messages [ 2754.461638] Lustre: 96992:0:(ldlm_lib.c:2070:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2754.481790] Lustre: 96992:0:(ldlm_lib.c:2070:extend_recovery_timer()) Skipped 6 previous similar messages [ 2755.126979] LustreError: 96992:0:(ldlm_lib.c:2683:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2755.137995] LustreError: 96992:0:(ldlm_lib.c:2683:replay_request_or_update()) Skipped 6 previous similar messages [ 2755.479618] LustreError: 96992:0:(ldlm_lib.c:2683:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2755.488978] LustreError: 96992:0:(ldlm_lib.c:2683:replay_request_or_update()) Skipped 6 previous similar messages [ 2758.669251] Lustre: 96992:0:(ldlm_lib.c:2070:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2758.696120] Lustre: 96992:0:(ldlm_lib.c:2070:extend_recovery_timer()) Skipped 12 previous similar messages [ 2759.337897] LustreError: 96992:0:(ldlm_lib.c:2683:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2759.352082] LustreError: 96992:0:(ldlm_lib.c:2683:replay_request_or_update()) Skipped 12 previous similar messages [ 2759.671252] LustreError: 96992:0:(ldlm_lib.c:2683:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2759.679543] LustreError: 96992:0:(ldlm_lib.c:2683:replay_request_or_update()) Skipped 12 previous similar messages [ 2766.709937] Lustre: 96992:0:(ldlm_lib.c:2070:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2766.726185] Lustre: 96992:0:(ldlm_lib.c:2070:extend_recovery_timer()) Skipped 24 previous similar messages [ 2767.367040] LustreError: 96992:0:(ldlm_lib.c:2683:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2767.383719] LustreError: 96992:0:(ldlm_lib.c:2683:replay_request_or_update()) Skipped 24 previous similar messages [ 2767.712663] LustreError: 96992:0:(ldlm_lib.c:2683:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2767.730143] LustreError: 96992:0:(ldlm_lib.c:2683:replay_request_or_update()) Skipped 24 previous similar messages [ 2782.943967] Lustre: 96992:0:(ldlm_lib.c:2070:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2782.952167] Lustre: 96992:0:(ldlm_lib.c:2070:extend_recovery_timer()) Skipped 50 previous similar messages [ 2783.584317] LustreError: 96992:0:(ldlm_lib.c:2683:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2783.602196] LustreError: 96992:0:(ldlm_lib.c:2683:replay_request_or_update()) Skipped 50 previous similar messages [ 2783.927113] LustreError: 96992:0:(ldlm_lib.c:2683:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2783.936075] LustreError: 96992:0:(ldlm_lib.c:2683:replay_request_or_update()) Skipped 50 previous similar messages [ 2810.199266] Lustre: 96992:0:(ldlm_lib.c:2379:target_recovery_overseer()) lustre-MDT0000 recovery is aborted by hard timeout [ 2810.210902] Lustre: 96992:0:(ldlm_lib.c:2389:target_recovery_overseer()) recovery is aborted, evict exports in recovery [ 2810.226671] Lustre: 96992:0:(genops.c:1622:class_disconnect_stale_exports()) lustre-MDT0000: disconnect stale client ded1baa9-02ef-4290-9545-afc7657f274b@192.168.203.58@tcp [ 2810.259170] Lustre: lustre-MDT0000: disconnecting 1 stale clients [ 2810.269879] Lustre: 96992:0:(ldlm_lib.c:1899:abort_req_replay_queue()) @@@ aborted: req@00000000b7ae0153 x1863839292398976/t0(8589934782) o35->ded1baa9-02ef-4290-9545-afc7657f274b@192.168.203.58@tcp:193/0 lens 392/0 e 0 to 0 dl 1777496693 ref 1 fl Complete:/604/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:0 [ 2810.304184] Lustre: lustre-MDT0000-osd: cancel update llog [0x200000400:0x1:0x0] [ 2810.326099] Lustre: lustre-MDT0000: Denying connection for new client ded1baa9-02ef-4290-9545-afc7657f274b (at 192.168.203.58@tcp), waiting for 3 known clients (2 recovered, 0 in progress, and 1 evicted) to recover in 0:00 [ 2810.353266] Lustre: lustre-MDT0001-osp-MDT0000: cancel update llog [0x240000401:0x1:0x0] [ 2810.396666] Lustre: lustre-MDT0000: Recovery over after 1:00, of 3 clients 2 recovered and 1 was evicted. [ 2810.397818] Lustre: lustre-MDT0000-osp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 2810.437709] Lustre: Skipped 2 previous similar messages [ 2810.475968] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:49 to 0x2c0000401:545) [ 2810.493472] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:80 to 0x280000401:577) [ 2825.184516] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2825.195777] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2825.218471] Lustre: Skipped 3 previous similar messages [ 2825.231486] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2825.238357] Lustre: Skipped 3 previous similar messages [ 2830.192542] Lustre: server umount lustre-OST0000 complete [ 2841.682711] Lustre: server umount lustre-OST0001 complete [ 2853.867623] Lustre: server umount lustre-MDT0000 complete [ 2856.417548] LustreError: 91270:0:(ldlm_lib.c:1180:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2856.445395] LustreError: 91270:0:(ldlm_lib.c:1180:target_handle_connect()) Skipped 48 previous similar messages [ 2859.075276] LustreError: 91252:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1777496731 with bad export cookie 2244971675758631931 [ 2859.076769] LustreError: MGC192.168.203.158@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2859.098216] LustreError: 91252:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 2859.615074] Lustre: server umount lustre-MDT0001 complete [ 2869.792818] Lustre: DEBUG MARKER: == conf-sanity test 85: osd_ost init: fail ea_fid_set ==== 17:05:40 (1777496740) [ 2878.571245] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2878.592683] Lustre: *** cfs_fail_loc=197, val=0*** [ 2894.303830] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 2899.424224] LustreError: 98953:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.203.158@tcp: failed processing log, type 4: rc = -110 [ 2904.598133] Lustre: *** cfs_fail_loc=197, val=0*** [ 2904.601938] Lustre: Skipped 65 previous similar messages [ 2925.153808] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2925.169294] Lustre: Skipped 2 previous similar messages [ 2932.904332] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing set_default_debug -1 all [ 2939.239917] Lustre: DEBUG MARKER: oleg358-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2941.810160] Lustre: server umount lustre-OST0000 complete [ 2951.150453] Lustre: DEBUG MARKER: == conf-sanity test 86: Replacing mkfs.lustre -G option == 17:07:01 (1777496821) [ 2973.059325] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2978.549830] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing set_default_debug -1 all [ 2986.850889] Lustre: DEBUG MARKER: oleg358-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2991.244193] LustreError: 100502:0:(osp_object.c:617:osp_attr_get()) lustre-MDT0001-osp-MDT0000: osp_attr_get update error [0x200000009:0x1:0x0]: rc = -5 [ 2991.256068] LustreError: 100502:0:(lod_sub_object.c:917:lod_sub_prep_llog()) lustre-MDT0000-mdtlov: can't get id from catalogs: rc = -5 [ 2991.264375] LustreError: 100502:0:(lod_dev.c:510:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 18, retries 0, failed: rc = -5 [ 2991.834790] Lustre: server umount lustre-MDT0000 complete [ 3029.082632] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing set_hostid [ 3040.624092] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing load_modules_local [ 3083.059343] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3083.279981] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3083.332278] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3083.401512] Lustre: lustre-MDT0000: new disk, initializing [ 3083.488158] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3087.829960] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing set_default_debug -1 all [ 3099.196471] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3099.333879] Lustre: 104531:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 3099.584589] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3104.058834] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing set_default_debug -1 all [ 3113.573914] Lustre: DEBUG MARKER: oleg358-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 3117.285419] Lustre: DEBUG MARKER: oleg358-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 3124.511884] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3131.846760] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing set_default_debug -1 all [ 3132.629068] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3140.219851] Lustre: DEBUG MARKER: oleg358-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3145.045883] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 3145.268204] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 3149.667875] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 3149.886897] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 3153.377926] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3153.383029] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3153.400524] LustreError: Skipped 4 previous similar messages [ 3153.401290] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3153.401295] Lustre: Skipped 11 previous similar messages [ 3153.425339] Lustre: Skipped 6 previous similar messages [ 3157.881735] Lustre: server umount lustre-OST0000 complete [ 3168.588353] Lustre: server umount lustre-MDT0000 complete [ 3168.739667] LustreError: 105660:0:(ldlm_lib.c:1180:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3168.773714] LustreError: 105660:0:(ldlm_lib.c:1180:target_handle_connect()) Skipped 5 previous similar messages [ 3172.378337] LustreError: 106271:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1777497045 with bad export cookie 2244971675758642655 [ 3172.383818] LustreError: MGC192.168.203.158@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3172.407157] LustreError: 106271:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 3172.951619] Lustre: server umount lustre-MDT0001 complete [ 3182.512371] Lustre: DEBUG MARKER: == conf-sanity test 87: check if MDT inode can hold EAs with N stripes properly ========================================================== 17:10:53 (1777497053) [ 3192.990560] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing unload_modules_local [ 3197.452653] Key type lgssc unregistered [ 3197.850622] LNet: 108593:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3197.857064] LNetError: 108593:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3197.870771] LNet: Removed LNI 192.168.203.158@tcp [ 3198.837928] Key type .llcrypt unregistered [ 3198.840950] Key type ._llcrypt unregistered [ 3218.850734] Key type ._llcrypt registered [ 3218.853975] Key type .llcrypt registered [ 3218.988302] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing set_hostid [ 3233.703178] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing load_modules_local [ 3235.028279] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3235.042217] alg: No test for adler32 (adler32-zlib) [ 3236.140393] Lustre: Lustre: Build Version: 2.17.52_93_g6cdb58a [ 3236.398841] LNet: Added LNI 192.168.203.158@tcp [8/256/0/180] [ 3238.119673] Key type lgssc registered [ 3239.179486] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3280.292573] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3280.318486] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3281.657757] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3281.718964] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3281.850782] Lustre: lustre-MDT0000: new disk, initializing [ 3281.989874] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3282.020172] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3286.388463] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing set_default_debug -1 all [ 3297.537080] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3297.854677] Lustre: lustre-OST0000: new disk, initializing [ 3297.862276] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3297.867608] Lustre: Skipped 1 previous similar message [ 3297.944199] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3303.733351] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing set_default_debug -1 all [ 3304.050304] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 3304.064985] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 3304.102063] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 3313.263515] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3313.488204] Lustre: lustre-OST0001: new disk, initializing [ 3313.493646] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 3313.591916] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 3319.942454] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing set_default_debug -1 all [ 3321.479260] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:1:ost [ 3321.488596] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:1:ost] [ 3321.525797] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x280000400 [ 3332.071807] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3332.080306] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3332.095708] Lustre: Skipped 1 previous similar message [ 3332.112504] Lustre: Skipped 1 previous similar message [ 3335.927798] Lustre: server umount lustre-MDT0000 complete [ 3342.384060] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: (null) [ 3347.492957] LustreError: 111929:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1777497220 with bad export cookie 7635364251080710310 [ 3347.506293] LustreError: MGC192.168.203.158@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3347.523675] LustreError: 111929:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 3347.834035] Lustre: server umount lustre-OST0000 complete [ 3352.219404] Lustre: server umount lustre-OST0001 complete [ 3360.638206] Lustre: DEBUG MARKER: == conf-sanity test 88: check the default mount options can be overridden ========================================================== 17:13:51 (1777497231) [ 3370.072141] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing load_modules_local [ 3392.903831] Lustre: DEBUG MARKER: == conf-sanity test 89: check tunefs --param and --erase-param{s} options ========================================================== 17:14:23 (1777497263) [ 3429.057405] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing set_hostid [ 3437.464417] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing load_modules_local [ 3472.028118] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3472.362261] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3472.423582] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3472.513942] Lustre: lustre-MDT0000: new disk, initializing [ 3472.638695] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3472.656976] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3477.677141] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing set_default_debug -1 all [ 3490.508325] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3490.604274] Lustre: 120253:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 3490.681762] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 3490.693465] Lustre: Skipped 1 previous similar message [ 3490.872722] Lustre: lustre-MDT0001: new disk, initializing [ 3491.085745] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3491.163479] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 3491.200583] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3495.816912] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing set_default_debug -1 all [ 3503.822936] Lustre: DEBUG MARKER: oleg358-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 3507.956302] Lustre: DEBUG MARKER: oleg358-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 3514.861380] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3515.199700] Lustre: lustre-OST0000: new disk, initializing [ 3515.207977] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3515.335241] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3521.053730] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing set_default_debug -1 all [ 3522.662678] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 3522.680954] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 3522.804320] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3528.845139] Lustre: DEBUG MARKER: oleg358-client.virtnet: executing wait_import_state_mount (FULL|IDLE) failover.node=192.0.2.254@tcp0,osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3534.073867] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 3534.410968] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 3538.997586] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 3539.277701] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 3543.522366] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3543.525386] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3543.537077] LustreError: Skipped 1 previous similar message [ 3543.568080] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3548.640782] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3548.652794] Lustre: Skipped 2 previous similar messages [ 3553.772211] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3555.298250] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 3555.429984] Lustre: server umount lustre-OST0000 complete [ 3558.886921] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3558.892283] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3558.900711] Lustre: Skipped 2 previous similar messages [ 3558.923491] Lustre: Skipped 2 previous similar messages [ 3569.123682] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3569.135813] Lustre: Skipped 3 previous similar messages [ 3573.216033] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 3573.543174] Lustre: server umount lustre-MDT0000 complete [ 3577.540141] LustreError: 120247:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1777497450 with bad export cookie 7635364251080711535 [ 3577.549090] LustreError: MGC192.168.203.158@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3577.570394] LustreError: 120247:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 3578.102795] Lustre: server umount lustre-MDT0001 complete [ 3587.150105] Lustre: DEBUG MARKER: == conf-sanity test 90a: check max_mod_rpcs_in_flight is enforced ========================================================== 17:17:37 (1777497457) [ 3595.631781] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3596.520905] LustreError: 124265:0:(ldlm_lib.c:1180:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3596.555271] LustreError: 124265:0:(ldlm_lib.c:1180:target_handle_connect()) Skipped 3 previous similar messages [ 3596.607297] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3601.889692] LustreError: 124266:0:(ldlm_lib.c:1180:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3602.408634] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing set_default_debug -1 all [ 3607.008777] LustreError: 124265:0:(ldlm_lib.c:1180:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3612.133940] LustreError: 124266:0:(ldlm_lib.c:1180:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3613.683555] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3620.204668] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing set_default_debug -1 all [ 3626.530426] Lustre: DEBUG MARKER: oleg358-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 3631.982644] Lustre: DEBUG MARKER: oleg358-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 3638.203192] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3638.629378] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3638.632074] Lustre: Skipped 1 previous similar message [ 3644.241663] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing set_default_debug -1 all [ 3649.632064] Lustre: DEBUG MARKER: oleg358-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3653.852214] Lustre: *** cfs_fail_loc=159, val=0*** [ 3670.452240] Lustre: lustre-MDT0000: Client 9208d51d-eb84-40ca-b8d2-1f721271c79c (at 192.168.203.58@tcp) reconnecting [ 3672.279497] Lustre: *** cfs_fail_loc=159, val=0*** [ 3672.285537] Lustre: Skipped 5 previous similar messages [ 3687.895774] Lustre: lustre-MDT0000: Client 9208d51d-eb84-40ca-b8d2-1f721271c79c (at 192.168.203.58@tcp) reconnecting [ 3706.341464] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3706.357677] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3706.375701] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3708.384367] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3712.377549] Lustre: server umount lustre-OST0000 complete [ 3716.577374] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3716.584334] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3716.608893] Lustre: Skipped 1 previous similar message [ 3722.642460] Lustre: server umount lustre-MDT0000 complete [ 3723.746602] LustreError: 126465:0:(ldlm_lib.c:1180:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3723.757285] LustreError: 126465:0:(ldlm_lib.c:1180:target_handle_connect()) Skipped 1 previous similar message [ 3726.366203] LustreError: 126651:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1777497599 with bad export cookie 7635364251080712417 [ 3726.384172] LustreError: MGC192.168.203.158@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3726.654828] Lustre: server umount lustre-MDT0001 complete [ 3738.350821] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing unload_modules_local [ 3741.516863] Key type lgssc unregistered [ 3741.891895] LNet: 127688:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3741.901622] LNetError: 127688:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3741.945206] LNet: Removed LNI 192.168.203.158@tcp [ 3742.806177] Key type .llcrypt unregistered [ 3742.808094] Key type ._llcrypt unregistered [ 3764.752727] Key type ._llcrypt registered [ 3764.755902] Key type .llcrypt registered [ 3764.876652] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing load_modules_local [ 3765.935980] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3765.985471] alg: No test for adler32 (adler32-zlib) [ 3767.114040] Lustre: Lustre: Build Version: 2.17.52_93_g6cdb58a [ 3767.352971] LNet: Added LNI 192.168.203.158@tcp [8/256/0/180] [ 3769.047176] Key type lgssc registered [ 3770.166799] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3779.180473] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3779.259847] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3780.656305] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3785.669933] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing set_default_debug -1 all [ 3790.824309] LustreError: 129086:0:(ldlm_lib.c:1180:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3790.842504] LustreError: 129086:0:(ldlm_lib.c:1180:target_handle_connect()) Skipped 2 previous similar messages [ 3795.937555] LustreError: 129085:0:(ldlm_lib.c:1180:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3796.388174] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3796.978550] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3801.203059] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing set_default_debug -1 all [ 3807.012173] Lustre: DEBUG MARKER: oleg358-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 3811.763219] Lustre: DEBUG MARKER: oleg358-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 3818.466466] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3818.940236] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3825.155361] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing set_default_debug -1 all [ 3826.122378] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:42 to 0x280000401:65) [ 3831.450575] Lustre: DEBUG MARKER: oleg358-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3841.473937] Lustre: *** cfs_fail_loc=159, val=0*** [ 3856.863332] Lustre: lustre-MDT0000: Client e7405938-f7c7-4883-9ea6-fb86883c980a (at 192.168.203.58@tcp) reconnecting [ 3859.657832] Lustre: *** cfs_fail_loc=159, val=0*** [ 3859.670243] Lustre: Skipped 2 previous similar messages [ 3875.295993] Lustre: lustre-MDT0001: Client e7405938-f7c7-4883-9ea6-fb86883c980a (at 192.168.203.58@tcp) reconnecting [ 3876.894620] Lustre: *** cfs_fail_loc=159, val=0*** [ 3893.198325] Lustre: lustre-MDT0001: Client e7405938-f7c7-4883-9ea6-fb86883c980a (at 192.168.203.58@tcp) reconnecting [ 3898.829234] Lustre: 131793:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 3902.564238] Lustre: *** cfs_fail_loc=159, val=0*** [ 3902.568666] Lustre: Skipped 5 previous similar messages [ 3917.847567] Lustre: lustre-MDT0000: Client 25b6ba98-f1b7-4083-ab24-0f3738f52ac1 (at 192.168.203.58@tcp) reconnecting [ 3919.587303] Lustre: *** cfs_fail_loc=159, val=0*** [ 3919.602819] Lustre: Skipped 13 previous similar messages [ 3935.175926] Lustre: lustre-MDT0000: Client 25b6ba98-f1b7-4083-ab24-0f3738f52ac1 (at 192.168.203.58@tcp) reconnecting [ 3936.907684] Lustre: 132085:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 3943.918954] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3943.937311] Lustre: Skipped 1 previous similar message [ 3943.939080] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3943.945867] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3943.966935] Lustre: Skipped 1 previous similar message [ 3949.024083] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3949.037402] Lustre: Skipped 1 previous similar message [ 3949.623651] Lustre: server umount lustre-OST0000 complete [ 3954.147218] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3954.152375] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3954.154117] Lustre: Skipped 1 previous similar message [ 3954.161549] Lustre: Skipped 1 previous similar message [ 3959.264930] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3959.270754] Lustre: Skipped 1 previous similar message [ 3959.747103] Lustre: server umount lustre-MDT0000 complete [ 3963.812759] LustreError: 129066:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1777497836 with bad export cookie 10363344150654033612 [ 3963.814616] LustreError: MGC192.168.203.158@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3963.820202] LustreError: 129066:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 3964.215713] Lustre: server umount lustre-MDT0001 complete [ 3975.706244] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing unload_modules_local [ 3978.489580] Key type lgssc unregistered [ 3978.757047] LNet: 133122:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3978.767809] LNetError: 133122:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3978.793869] LNet: Removed LNI 192.168.203.158@tcp [ 3979.791742] Key type .llcrypt unregistered [ 3979.794874] Key type ._llcrypt unregistered [ 4004.944859] Key type ._llcrypt registered [ 4004.947437] Key type .llcrypt registered [ 4005.110304] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing load_modules_local [ 4006.160906] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4006.197413] alg: No test for adler32 (adler32-zlib) [ 4007.370744] Lustre: Lustre: Build Version: 2.17.52_93_g6cdb58a [ 4007.687629] LNet: Added LNI 192.168.203.158@tcp [8/256/0/180] [ 4009.368775] Key type lgssc registered [ 4010.516742] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4018.023391] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4018.054370] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4019.756636] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4024.325608] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing set_default_debug -1 all [ 4029.926390] LustreError: 134521:0:(ldlm_lib.c:1180:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4029.940876] LustreError: 134521:0:(ldlm_lib.c:1180:target_handle_connect()) Skipped 2 previous similar messages [ 4032.135487] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4032.640960] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4037.905883] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing set_default_debug -1 all [ 4043.516652] Lustre: DEBUG MARKER: oleg358-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 4047.635361] Lustre: DEBUG MARKER: oleg358-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 4054.274346] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4054.577202] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4057.663431] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:97) [ 4060.416598] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing set_default_debug -1 all [ 4064.849971] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:129) [ 4065.579366] Lustre: DEBUG MARKER: oleg358-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4071.118179] Lustre: 136763:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4074.301293] Lustre: 136812:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4075.849567] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 in log params [ 4076.863554] Lustre: Setting parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=8 in log params [ 4080.644946] Lustre: 136959:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify lustre/mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 (mode = 0) failed: rc = -17 [ 4081.491125] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=7 in log params [ 4082.420809] Lustre: 137067:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4088.288137] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4088.294806] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4088.310659] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4090.338596] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4090.345277] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4092.999800] Lustre: server umount lustre-OST0000 complete [ 4099.040490] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4099.052086] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4099.075554] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4099.083979] Lustre: Skipped 1 previous similar message [ 4105.696794] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4105.699284] Lustre: Skipped 3 previous similar messages [ 4110.817735] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4110.824794] Lustre: Skipped 1 previous similar message [ 4111.327248] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 4111.662139] Lustre: server umount lustre-MDT0000 complete [ 4115.653886] LustreError: 136861:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1777497988 with bad export cookie 7481752794811815471 [ 4115.661053] LustreError: MGC192.168.203.158@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4115.668845] LustreError: 136861:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 4116.081184] Lustre: server umount lustre-MDT0001 complete [ 4126.980741] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing unload_modules_local [ 4129.889931] Key type lgssc unregistered [ 4130.325545] LNet: 138101:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4130.341393] LNetError: 138101:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4130.378276] LNet: Removed LNI 192.168.203.158@tcp [ 4131.207861] Key type .llcrypt unregistered [ 4131.210213] Key type ._llcrypt unregistered [ 4158.078284] Key type ._llcrypt registered [ 4158.086135] Key type .llcrypt registered [ 4158.185867] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing load_modules_local [ 4159.421144] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4159.519562] alg: No test for adler32 (adler32-zlib) [ 4160.594523] Lustre: Lustre: Build Version: 2.17.52_93_g6cdb58a [ 4160.792365] LNet: Added LNI 192.168.203.158@tcp [8/256/0/180] [ 4162.495426] Key type lgssc registered [ 4163.642327] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4171.504755] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4171.527446] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4173.142136] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4177.016133] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing set_default_debug -1 all [ 4183.522376] LustreError: 139501:0:(ldlm_lib.c:1180:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4183.547842] LustreError: 139501:0:(ldlm_lib.c:1180:target_handle_connect()) Skipped 2 previous similar messages [ 4185.097297] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4185.537386] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4189.260483] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing set_default_debug -1 all [ 4193.889659] Lustre: DEBUG MARKER: oleg358-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 4198.304433] Lustre: DEBUG MARKER: oleg358-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 4204.641173] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4204.979448] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4211.124766] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:129) [ 4211.567111] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing set_default_debug -1 all [ 4217.856240] Lustre: DEBUG MARKER: oleg358-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4218.299902] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:161) [ 4224.029272] Lustre: *** cfs_fail_loc=159, val=0*** [ 4240.320580] Lustre: lustre-MDT0000: Client 7881ac31-650b-4ee0-b952-64255541acfc (at 192.168.203.58@tcp) reconnecting [ 4246.005665] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4246.005560] LustreError: lustre-OST0000-osc-MDT0000: operation ost_destroy to node 0@lo failed: rc = -107 [ 4246.033594] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4246.038792] LustreError: Skipped 6 previous similar messages [ 4247.010012] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4247.021585] LustreError: Skipped 1 previous similar message [ 4247.027568] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4247.036269] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4247.289951] Lustre: server umount lustre-OST0000 complete [ 4252.136966] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4252.151797] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4252.171955] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4254.176215] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4254.180795] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4257.324095] Lustre: server umount lustre-MDT0000 complete [ 4259.301426] LustreError: 141708:0:(ldlm_lib.c:1180:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4259.312795] LustreError: 141708:0:(ldlm_lib.c:1180:target_handle_connect()) Skipped 1 previous similar message [ 4260.994895] LustreError: 139482:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1777498133 with bad export cookie 4519472942513389907 [ 4261.003374] LustreError: MGC192.168.203.158@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4261.010908] LustreError: 139482:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 4261.429637] Lustre: server umount lustre-MDT0001 complete [ 4273.625037] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing unload_modules_local [ 4276.456359] Key type lgssc unregistered [ 4276.814632] LNet: 142809:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4276.828354] LNetError: 142809:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4277.866061] LNet: Removed LNI 192.168.203.158@tcp [ 4278.634164] Key type .llcrypt unregistered [ 4278.636246] Key type ._llcrypt unregistered [ 4299.502804] Key type ._llcrypt registered [ 4299.505302] Key type .llcrypt registered [ 4299.609483] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing load_modules_local [ 4300.611382] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4300.721743] alg: No test for adler32 (adler32-zlib) [ 4301.734336] Lustre: Lustre: Build Version: 2.17.52_93_g6cdb58a [ 4301.964442] LNet: Added LNI 192.168.203.158@tcp [8/256/0/180] [ 4303.671238] Key type lgssc registered [ 4304.864686] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4311.989238] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4312.012050] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4313.450857] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4317.395679] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing set_default_debug -1 all [ 4323.813869] LustreError: 144203:0:(ldlm_lib.c:1180:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4323.840188] LustreError: 144203:0:(ldlm_lib.c:1180:target_handle_connect()) Skipped 2 previous similar messages [ 4326.141936] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4326.693842] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4331.051280] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing set_default_debug -1 all [ 4336.177892] Lustre: DEBUG MARKER: oleg358-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 4340.164302] Lustre: DEBUG MARKER: oleg358-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 4345.966331] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4346.361287] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4351.130594] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing set_default_debug -1 all [ 4351.480813] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:161) [ 4355.486527] Lustre: DEBUG MARKER: oleg358-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4358.639582] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:170 to 0x280000401:193) [ 4377.160696] Lustre: lustre-MDT0001: evicting a51b146a-7c0c-4d00-b3ad-6f2fcdbe6c9d (at 192.168.203.58@tcp) by administrative request [ 4377.171028] Lustre: Skipped 1 previous similar message [ 4404.704670] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4404.705925] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4404.709679] Lustre: Skipped 1 previous similar message [ 4404.709962] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4409.832592] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4409.838034] Lustre: Skipped 2 previous similar messages [ 4410.211092] Lustre: server umount lustre-OST0000 complete [ 4413.922056] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4413.927213] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4413.943610] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4414.947862] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4419.518546] Lustre: server umount lustre-MDT0000 complete [ 4420.067343] LustreError: 144204:0:(ldlm_lib.c:1180:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4420.080053] LustreError: 144204:0:(ldlm_lib.c:1180:target_handle_connect()) Skipped 1 previous similar message [ 4422.706376] LustreError: 144183:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1777498295 with bad export cookie 9722006084076448850 [ 4422.710543] LustreError: MGC192.168.203.158@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4422.714581] LustreError: 144183:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 4423.008939] Lustre: server umount lustre-MDT0001 complete [ 4431.838655] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing unload_modules_local [ 4434.211742] Key type lgssc unregistered [ 4434.431938] LNet: 147926:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4434.440608] LNetError: 147926:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4434.451297] LNet: Removed LNI 192.168.203.158@tcp [ 4435.044208] Key type .llcrypt unregistered [ 4435.045665] Key type ._llcrypt unregistered [ 4488.202340] Key type ._llcrypt registered [ 4488.206353] Key type .llcrypt registered [ 4488.300409] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing load_modules_local [ 4489.083558] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4489.100635] alg: No test for adler32 (adler32-zlib) [ 4490.166976] Lustre: Lustre: Build Version: 2.17.52_93_g6cdb58a [ 4490.321194] LNet: Added LNI 192.168.203.158@tcp [8/256/0/180] [ 4491.935251] Key type lgssc registered [ 4492.816406] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4498.475906] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4498.509419] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4499.874635] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4503.451988] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing set_default_debug -1 all [ 4509.156134] LustreError: 150372:0:(ldlm_lib.c:1180:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4509.170491] LustreError: 150372:0:(ldlm_lib.c:1180:target_handle_connect()) Skipped 2 previous similar messages [ 4510.056293] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4510.409507] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4513.438158] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing set_default_debug -1 all [ 4517.641966] Lustre: DEBUG MARKER: oleg358-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 4520.234851] Lustre: DEBUG MARKER: oleg358-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 4525.227619] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4525.533080] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4529.497233] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing set_default_debug -1 all [ 4533.291422] Lustre: DEBUG MARKER: oleg358-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4535.802272] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:193) [ 4545.019692] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:195 to 0x280000401:225) [ 4556.258347] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4556.261952] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4556.268779] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4560.354448] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4560.355374] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4560.375115] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4561.868660] Lustre: server umount lustre-OST0000 complete [ 4565.474111] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4565.481378] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4565.484536] Lustre: Skipped 2 previous similar messages [ 4570.594788] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4570.611920] Lustre: Skipped 1 previous similar message [ 4570.926617] Lustre: server umount lustre-MDT0000 complete [ 4574.144805] LustreError: 150355:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1777498447 with bad export cookie 14729139610357814654 [ 4574.148242] LustreError: MGC192.168.203.158@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4574.161248] LustreError: 150355:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4574.376770] Lustre: server umount lustre-MDT0001 complete [ 4582.130935] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing unload_modules_local [ 4584.212672] Key type lgssc unregistered [ 4584.425331] LNet: 153579:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4584.431793] LNetError: 153579:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4584.444913] LNet: Removed LNI 192.168.203.158@tcp [ 4585.000742] Key type .llcrypt unregistered [ 4585.002691] Key type ._llcrypt unregistered [ 4603.401754] Key type ._llcrypt registered [ 4603.405838] Key type .llcrypt registered [ 4603.496383] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing load_modules_local [ 4604.320794] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4604.420072] alg: No test for adler32 (adler32-zlib) [ 4605.420753] Lustre: Lustre: Build Version: 2.17.52_93_g6cdb58a [ 4605.640489] LNet: Added LNI 192.168.203.158@tcp [8/256/0/180] [ 4607.319169] Key type lgssc registered [ 4608.302987] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4619.265872] Lustre: DEBUG MARKER: == conf-sanity test 100: check lshowmount lists MGS, MDT, OST and 0@lo ========================================================== 17:34:50 (1777498490) [ 4630.693682] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing set_hostid [ 4637.232920] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing load_modules_local [ 4662.446760] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4662.464933] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4663.656700] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 4663.684108] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 4663.734520] Lustre: lustre-MDT0000: new disk, initializing [ 4663.809721] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4663.830242] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 4666.625906] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing set_default_debug -1 all [ 4674.685039] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4674.775833] Lustre: 157770:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 4674.811697] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 4674.815211] Lustre: Skipped 1 previous similar message [ 4674.865383] Lustre: lustre-MDT0001: new disk, initializing [ 4674.943873] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4674.975171] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 4674.981640] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 4678.086771] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing set_default_debug -1 all [ 4683.728145] Lustre: DEBUG MARKER: oleg358-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 4686.201490] Lustre: DEBUG MARKER: oleg358-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 4690.904510] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4691.164868] Lustre: lustre-OST0000: new disk, initializing [ 4691.175832] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 4691.263122] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4695.398860] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing set_default_debug -1 all [ 4698.651252] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 4698.658610] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 4698.730544] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 4700.311282] Lustre: DEBUG MARKER: oleg358-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4708.834460] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4708.837026] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4708.840613] LustreError: Skipped 1 previous similar message [ 4708.843182] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4708.851413] Lustre: Skipped 1 previous similar message [ 4708.865468] Lustre: Skipped 1 previous similar message [ 4711.914499] Lustre: server umount lustre-OST0000 complete [ 4713.953417] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4713.955821] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4713.968115] Lustre: Skipped 1 previous similar message [ 4713.975576] Lustre: Skipped 1 previous similar message [ 4719.073023] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4719.076301] Lustre: Skipped 1 previous similar message [ 4724.192378] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4724.195424] Lustre: Skipped 1 previous similar message [ 4728.287861] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 4728.291887] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4728.316824] Lustre: Skipped 1 previous similar message [ 4728.426907] Lustre: server umount lustre-MDT0000 complete [ 4730.890084] LustreError: 157762:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1777498603 with bad export cookie 15753679026499608739 [ 4730.891398] LustreError: MGC192.168.203.158@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4730.899834] LustreError: 157762:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 4731.134305] Lustre: server umount lustre-MDT0001 complete [ 4737.525666] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing unload_modules_local [ 4739.186146] Key type lgssc unregistered [ 4739.406570] LNet: 161283:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4739.410972] LNetError: 161283:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4739.418778] LNet: Removed LNI 192.168.203.158@tcp [ 4739.901264] Key type .llcrypt unregistered [ 4739.906027] Key type ._llcrypt unregistered [ 4754.984184] Key type ._llcrypt registered [ 4754.985832] Key type .llcrypt registered [ 4755.055974] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing load_modules_local [ 4755.565078] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4755.640816] alg: No test for adler32 (adler32-zlib) [ 4756.645678] Lustre: Lustre: Build Version: 2.17.52_93_g6cdb58a [ 4756.833300] LNet: Added LNI 192.168.203.158@tcp [8/256/0/180] [ 4758.463342] Key type lgssc registered [ 4759.227211] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4764.644179] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4764.656876] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4766.032811] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4768.809842] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing set_default_debug -1 all [ 4775.134848] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4775.417189] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4778.195707] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing set_default_debug -1 all [ 4782.041415] Lustre: DEBUG MARKER: oleg358-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 4784.976216] Lustre: DEBUG MARKER: oleg358-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 4789.783165] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4790.047775] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4792.108756] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:3 to 0x280000401:33) [ 4793.874712] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing set_default_debug -1 all [ 4798.012283] Lustre: DEBUG MARKER: oleg358-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4803.683764] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4803.708618] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4803.722918] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4803.729700] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4803.737866] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4804.420725] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4804.433239] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4804.443971] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4804.448830] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4804.460694] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4805.164393] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4805.750202] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4805.757064] Lustre: Skipped 1 previous similar message [ 4805.769787] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4805.779587] Lustre: Skipped 1 previous similar message [ 4805.783759] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4805.788275] Lustre: Skipped 1 previous similar message [ 4805.792272] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4805.808295] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4805.814801] Lustre: Skipped 1 previous similar message [ 4806.383134] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4806.971558] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4807.578675] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4808.108161] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4808.111763] Lustre: Skipped 3 previous similar messages [ 4808.125349] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4808.138465] Lustre: Skipped 3 previous similar messages [ 4808.145320] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4808.150325] Lustre: Skipped 3 previous similar messages [ 4808.153693] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4808.160450] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4808.165402] Lustre: Skipped 3 previous similar messages [ 4808.717863] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4809.301658] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4810.043113] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4810.660413] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4811.333521] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4811.895650] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4812.491529] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4812.495178] Lustre: Skipped 6 previous similar messages [ 4812.503856] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4812.513134] Lustre: Skipped 6 previous similar messages [ 4812.517096] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4812.523499] Lustre: Skipped 6 previous similar messages [ 4812.529193] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4812.541601] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4812.547421] Lustre: Skipped 6 previous similar messages [ 4813.120841] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4813.678837] LustreError: 162679:0:(osp_precreate.c:652:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 4813.685241] LustreError: 162679:0:(osp_precreate.c:1356:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 4813.687974] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4814.273930] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4814.917615] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4815.550294] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4816.182939] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4816.770644] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4817.445280] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4818.072827] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4818.693229] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4819.308437] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4820.022583] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4820.662903] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4820.668967] Lustre: Skipped 12 previous similar messages [ 4820.682948] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4820.691278] Lustre: Skipped 12 previous similar messages [ 4820.695794] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4820.700076] Lustre: Skipped 12 previous similar messages [ 4820.703901] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4820.712253] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4820.719553] Lustre: Skipped 12 previous similar messages [ 4821.302110] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4821.896763] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4822.481592] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4823.037201] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4823.570987] LustreError: 162679:0:(osp_precreate.c:652:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 4823.578848] LustreError: 162679:0:(osp_precreate.c:1356:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 4823.582246] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4824.120847] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4824.657193] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4825.211448] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4825.793575] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4826.430726] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4827.031803] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4827.609911] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4828.203089] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4828.766326] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4829.331181] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4829.848126] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4830.364766] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4830.914941] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4831.530418] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4832.098467] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4832.654746] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4833.248398] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4833.838559] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4834.352069] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4834.958586] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4835.507722] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4836.114193] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4836.787936] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4836.791730] Lustre: Skipped 27 previous similar messages [ 4836.810807] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4836.823680] Lustre: Skipped 27 previous similar messages [ 4836.829318] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4836.836459] Lustre: Skipped 27 previous similar messages [ 4836.840138] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4836.854728] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4836.861131] Lustre: Skipped 27 previous similar messages [ 4837.450053] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4838.072049] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4838.667832] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4839.326280] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4840.044476] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4840.661911] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4841.249664] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4841.796299] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4842.356409] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4842.948509] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4843.454460] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4844.006717] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4844.606185] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4845.163710] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4845.843719] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4846.407602] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4847.045681] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4847.678194] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4848.346967] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4848.893283] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4849.430456] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4850.001171] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4850.573481] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4851.197097] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4851.873561] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4852.653972] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4853.479078] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4854.241205] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4854.970542] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4855.660384] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4856.228431] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4856.922226] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4857.461391] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4858.047278] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4858.621346] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4859.272691] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4859.883639] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4860.549308] LustreError: 162679:0:(osp_precreate.c:652:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 4860.557719] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4860.560770] LustreError: 162679:0:(osp_precreate.c:1356:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 4861.213593] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4861.934817] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4862.540371] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4863.177547] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4863.785056] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4864.383666] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4865.033657] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4865.686587] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4866.363725] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4867.008439] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4867.658285] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4868.225080] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4868.792593] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4868.796429] Lustre: Skipped 50 previous similar messages [ 4868.808777] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4869.385224] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4869.391577] Lustre: Skipped 51 previous similar messages [ 4869.394891] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4869.400150] Lustre: Skipped 51 previous similar messages [ 4869.403130] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4869.412822] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4869.415724] Lustre: Skipped 51 previous similar messages [ 4869.989886] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4870.636492] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4871.289638] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4871.947249] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4872.575462] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4873.276270] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4873.887052] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4874.472328] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4875.024606] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4875.615402] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4876.217416] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4876.907792] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4877.505644] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4878.131287] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4878.778211] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4879.430577] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4880.041618] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4880.703117] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4881.302429] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4881.951775] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4882.552989] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4883.141936] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4883.719939] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4884.243743] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4884.799862] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4885.303932] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4885.830895] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4886.493719] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4887.095480] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4887.757550] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4888.380239] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4888.995280] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4889.542049] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4890.142365] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4890.733232] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4891.315714] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4891.865342] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4892.485701] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4893.063073] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4893.629865] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4894.198345] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4894.727183] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4895.267846] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4895.838217] LustreError: 162679:0:(osp_precreate.c:652:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 4895.844917] LustreError: 162679:0:(osp_precreate.c:1356:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 4895.850822] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4896.375824] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4896.961081] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4897.505189] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4898.062519] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4898.585345] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4899.202568] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4899.780174] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4900.323293] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4900.895287] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4901.494799] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4902.020577] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4902.636322] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4903.248301] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4903.837940] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4904.456856] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4905.122845] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4905.663712] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4906.340316] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4906.942206] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4907.592982] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4908.307585] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4909.015053] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4909.648058] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4910.249429] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4910.886262] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4911.512921] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4912.145684] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4912.738542] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4913.291244] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4913.907949] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4914.441762] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4915.029859] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4915.488777] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4915.989123] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4916.503707] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4917.037311] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4917.543483] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4918.038548] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4918.590990] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4919.031380] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4919.461856] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4919.931586] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4920.370857] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4920.847578] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4921.315169] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4921.823109] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4922.275454] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4922.748903] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4923.335439] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4923.810166] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4924.294697] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4924.753753] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4925.257507] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4925.804651] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4926.298289] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4926.774837] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4927.316363] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4927.835566] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4928.315863] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4928.776073] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4929.219425] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4929.699772] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4930.141284] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4930.571684] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4931.033066] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4931.572311] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4932.069422] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4932.567103] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4933.136706] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4933.141846] Lustre: Skipped 113 previous similar messages [ 4933.151103] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4933.645707] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4933.651547] Lustre: Skipped 113 previous similar messages [ 4933.654496] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4933.658041] Lustre: Skipped 113 previous similar messages [ 4933.660690] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4933.667397] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4933.670748] Lustre: Skipped 113 previous similar messages [ 4934.135292] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4934.697240] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4935.252511] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4935.785193] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4936.335755] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4936.892026] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4937.444449] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4938.007987] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4938.535805] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4939.090551] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4939.618109] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4940.095315] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4940.679652] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4941.207625] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4941.777574] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4942.324308] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4942.823843] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4943.285888] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4943.791429] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4944.346518] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4944.852747] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4945.352686] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4945.817929] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4946.263894] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4946.734152] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4947.241197] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4947.855101] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4948.463293] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4949.098618] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4949.644819] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4950.175426] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4950.748621] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4951.277756] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4951.837490] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4952.417198] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4953.038095] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4953.641121] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4954.202939] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4954.833302] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4955.369638] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4955.897457] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4956.454936] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4957.033391] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4957.555220] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4958.081497] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4958.547739] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4959.086121] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4959.562820] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4960.089233] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4960.586665] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4961.094789] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4961.585627] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4962.083941] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4962.611589] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4963.096128] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4963.639434] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4964.227866] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4964.839159] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4965.347804] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4965.927896] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4966.501225] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4967.082661] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4967.588477] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4968.135412] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4968.751323] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4969.534448] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4970.299457] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4971.342487] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4972.731781] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4975.806068] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4977.333698] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4979.119643] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4981.226910] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4982.614530] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4984.149300] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4985.207757] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4986.722697] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4988.332222] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4990.063959] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4990.883495] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4992.444737] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4993.152144] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4994.379067] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4995.366841] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4996.906815] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4998.120985] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4999.473504] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5001.145743] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5002.787926] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5004.428848] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5005.143361] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5006.133385] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5006.851928] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5007.752160] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5009.898828] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5011.270587] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5012.510188] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5013.973552] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5014.982754] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5016.319855] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5018.549959] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5019.443496] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5020.215894] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5020.967690] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5022.454139] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5024.010090] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5024.918401] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5026.263159] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5027.302654] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5028.415303] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5029.667795] LustreError: 162679:0:(osp_precreate.c:652:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5029.683324] LustreError: 162679:0:(osp_precreate.c:1356:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5029.714379] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5030.947067] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5032.118284] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5033.164230] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5034.384949] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5035.410141] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5036.388469] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5037.527208] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5038.563104] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5039.871430] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5040.891620] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5041.676854] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5042.652397] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5043.565276] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5044.569608] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5045.246845] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5046.192559] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5047.211012] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5048.089562] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5048.904230] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5049.736111] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5050.672901] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5051.708464] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5052.625864] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5053.533273] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5054.400494] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5055.415773] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5056.538239] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5057.487853] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5058.363508] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5059.257400] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5060.134760] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5061.033587] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5061.926514] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5061.938128] Lustre: Skipped 144 previous similar messages [ 5061.955375] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5061.967134] Lustre: Skipped 143 previous similar messages [ 5061.970721] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5061.975071] Lustre: Skipped 143 previous similar messages [ 5061.978403] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5061.986811] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5061.992329] Lustre: Skipped 143 previous similar messages [ 5062.955798] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5063.834290] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5064.678045] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5065.439584] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5066.251335] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5067.047606] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5067.830514] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5068.643102] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5069.462908] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5070.241916] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5070.953867] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5071.760139] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5072.403419] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5073.176850] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5074.012118] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5074.726212] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5075.649864] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5076.555758] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5077.392644] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5078.375640] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5079.190885] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5080.043916] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5080.890798] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5081.729987] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5082.455338] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5083.353407] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5084.104734] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5084.857216] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5085.632964] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5086.410530] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5087.288407] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5088.065599] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5088.970203] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5089.859841] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5090.722323] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5091.604891] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5092.323494] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5093.133962] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5093.844040] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5094.554828] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5095.453391] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5096.376124] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5097.358350] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5098.155100] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5099.075977] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5099.855367] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5100.537768] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5101.139139] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5101.871581] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5102.423929] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5103.148652] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5103.969966] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5104.754865] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5105.529098] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5106.281199] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5106.994192] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5107.737113] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5108.400108] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5109.007797] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5109.647572] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5110.200738] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5110.830259] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5111.521852] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5112.118214] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5112.720886] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5113.325414] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5113.969559] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5114.673889] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5115.422707] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5116.153220] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5116.849661] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5117.636463] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5118.372570] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5119.213994] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5120.008139] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5120.742385] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5121.465180] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5122.189251] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5122.851295] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5123.552268] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5124.228436] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5124.945166] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5125.641982] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5126.512367] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5127.213981] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5127.924136] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5128.604265] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5129.283485] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5129.882686] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5130.553835] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5131.139804] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5131.723338] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5132.371793] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5133.120128] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5133.816765] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5134.572214] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5135.267724] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5135.979440] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5136.624896] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5137.353774] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5137.978094] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5138.707720] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5139.521910] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5140.325550] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5141.197782] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5142.011691] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5142.696728] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5143.362305] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5143.928984] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5144.617362] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5145.281567] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5145.925566] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5146.667200] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5147.349245] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5147.944482] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5148.658402] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5149.435493] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5150.234804] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5150.927241] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5151.621868] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5152.348440] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5153.037048] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5153.740870] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5154.690609] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5155.560328] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5156.300478] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5157.053085] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5157.773936] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5158.523284] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5159.213891] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5159.971366] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5160.683920] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5161.311353] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5162.062884] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5162.812320] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5163.604502] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5164.310465] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5164.908057] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5165.628510] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5166.255178] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5167.011611] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5167.705705] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5168.343506] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5169.032828] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5169.701868] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5170.364707] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5171.071501] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5171.772909] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5172.515311] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5173.140572] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5173.869175] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5174.517777] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5175.161659] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5175.913351] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5176.741303] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5177.691718] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5178.646210] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5179.322282] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5179.937900] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5180.586562] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5181.410744] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5182.211219] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5182.950268] LustreError: 162679:0:(osp_precreate.c:652:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5182.955276] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5182.958815] LustreError: 162679:0:(osp_precreate.c:1356:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5183.758609] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5184.546449] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5185.224312] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5185.838926] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5186.539084] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5187.237723] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5187.904333] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5188.503184] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5189.170444] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5189.883666] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5190.625958] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5191.258533] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5191.870990] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5192.470219] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5193.039291] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5193.579310] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5194.129631] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5194.649332] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5195.227871] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5195.757167] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5196.359743] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5196.930995] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5197.538347] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5198.247849] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5198.955323] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5199.654983] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5200.413268] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5201.182819] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5201.916324] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5202.802979] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5203.725597] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5204.491689] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5205.253678] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5205.930312] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5206.842319] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5207.634581] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5208.311496] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5209.035372] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5209.834792] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5210.637092] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5211.381254] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5212.165155] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5212.898687] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5213.715235] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5214.463909] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5215.151376] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5215.894447] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5216.583913] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5217.447940] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5218.367896] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5219.267546] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5220.052973] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5220.908874] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5221.607146] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5222.476651] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5223.174547] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5223.917637] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5224.649148] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5225.342749] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5226.135109] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5226.953068] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5227.702158] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5228.471872] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5229.217868] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5229.910143] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5230.620236] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5231.409412] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5232.115717] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5232.884743] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5233.669565] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5234.487195] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5235.556346] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5236.621090] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5237.537488] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5238.366713] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5239.066413] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5239.988838] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5240.790895] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5241.469160] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5242.209716] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5243.076161] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5243.819740] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5244.692604] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5245.585443] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5246.430171] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5247.228777] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5248.122284] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5248.967692] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5249.784685] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5250.545945] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5251.286524] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5252.095828] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5252.910652] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5253.671898] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5254.454837] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5255.155556] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5255.804649] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5256.460679] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5257.225751] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5257.903347] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5258.682879] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5259.433140] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5260.127336] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5260.749771] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5261.342867] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5261.924714] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5262.597964] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5263.281087] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5263.992330] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5264.597720] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5265.278981] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5266.088270] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5266.899883] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5267.746792] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5268.538175] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5269.210418] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5269.891508] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5270.583890] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5271.263465] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5272.053931] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5272.864913] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5273.676431] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5274.404801] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5275.131555] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5275.904534] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5276.611954] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5277.312440] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5278.114284] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5278.934346] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5279.632381] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5280.311666] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5281.037572] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5281.712814] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5282.385952] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5283.020650] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5283.734477] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5284.432273] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5285.060038] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5285.674994] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5286.267845] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5286.938884] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5287.612286] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5288.332838] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5288.967768] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5289.561373] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5290.140648] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5290.784199] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5291.407545] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5292.017620] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5292.595291] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5293.233854] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5293.847179] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5294.421450] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5295.047453] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5295.596519] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5296.181964] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5296.801322] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5297.381415] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5297.951586] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5298.557973] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5299.196849] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5299.858754] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5300.580470] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5301.240514] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5301.795739] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5302.364389] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5302.985811] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5303.517946] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5304.030823] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5304.527529] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5305.062296] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5305.609177] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5306.140931] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5306.697493] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5307.290278] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5307.881915] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5308.457422] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5309.007782] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5309.562257] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5310.058736] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5310.680775] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5311.288712] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5311.957313] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5312.586489] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5313.228071] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5313.919732] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5314.586384] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5315.286329] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5315.970610] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5316.632104] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5317.249537] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5317.965165] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5317.974153] Lustre: Skipped 354 previous similar messages [ 5317.987872] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5318.001141] Lustre: Skipped 354 previous similar messages [ 5318.004787] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5318.010700] Lustre: Skipped 354 previous similar messages [ 5318.014189] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5318.030025] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5318.034814] Lustre: Skipped 354 previous similar messages [ 5318.632482] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5319.243335] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5319.846163] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5320.456438] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5321.088335] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5321.644724] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5322.234705] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5322.838809] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5323.447839] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5324.057814] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5324.668684] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5325.235373] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5325.756265] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5326.302879] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5326.894021] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5327.526114] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5328.131334] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5328.721158] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5329.239479] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5329.795593] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5330.366567] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5330.882643] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5331.464041] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5332.130862] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5332.744958] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5333.347316] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5333.892310] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5334.494381] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5335.022992] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5335.642063] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5336.184143] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5336.697575] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5337.258693] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5337.873527] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5338.449512] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5338.992394] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5339.565636] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5340.198168] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5340.742839] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5341.371964] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5341.920384] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5342.497785] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5343.117936] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5343.802455] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5344.416913] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5345.042040] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5345.689722] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5346.415434] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5347.093818] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5347.780863] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5348.451697] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5349.057872] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5349.690841] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5350.280432] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5350.872755] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5351.485446] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5352.082848] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5352.675334] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5353.354088] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5353.933755] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5354.465297] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5355.084816] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5355.796432] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5356.406482] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5357.120865] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5357.707417] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5358.363672] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5359.071356] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5359.692431] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5360.321055] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5360.907961] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5361.455496] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5362.043769] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5362.706080] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5363.434601] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5364.206543] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5365.023234] LustreError: 162679:0:(osp_precreate.c:652:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5365.033873] LustreError: 162679:0:(osp_precreate.c:1356:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5365.037326] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5365.750677] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5366.590341] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5367.242665] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5368.011765] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5368.622849] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5369.292606] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5369.968305] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5370.543854] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5371.126210] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5371.712163] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5372.317879] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5372.900973] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5373.500594] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5374.160050] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5374.745521] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5375.332393] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5375.945091] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5376.563733] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5377.169451] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5377.842939] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5378.475941] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5379.171715] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5379.899140] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5380.614377] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5381.328041] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5382.075494] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5382.740270] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5383.378815] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5384.060656] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5384.722800] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5385.319879] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5386.078709] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5386.785864] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5387.413084] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5388.180475] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5388.891591] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5389.567403] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5390.401117] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5391.068023] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5391.685467] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5392.297950] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5392.863282] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5393.503846] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5394.196515] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5394.974803] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5395.691816] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5396.368737] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5397.057364] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5397.713586] LustreError: 162679:0:(osp_precreate.c:652:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5397.720813] LustreError: 162679:0:(osp_precreate.c:652:osp_precreate_send()) Skipped 1 previous similar message [ 5397.726090] LustreError: 162679:0:(osp_precreate.c:1356:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5397.728415] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5397.733779] LustreError: 162679:0:(osp_precreate.c:1356:osp_precreate_thread()) Skipped 1 previous similar message [ 5398.282826] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5398.940530] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5399.515171] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5400.187436] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5400.773836] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5401.399486] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5401.971514] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5402.627213] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5403.207303] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5403.881793] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5404.538703] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5405.197603] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5405.786733] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5406.495640] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5407.073546] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5407.658633] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5408.260188] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5408.883108] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5409.605884] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5410.383719] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5411.124698] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5411.731223] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5412.308231] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5413.031478] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5413.687762] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5414.291806] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5414.931051] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5415.484444] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5416.207661] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5416.864936] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5417.397216] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5417.950968] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5418.483066] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5419.161459] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5419.709531] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5420.293612] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5420.977585] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5421.606898] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5422.283678] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5423.062596] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5423.714643] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5424.387331] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5424.991732] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5425.622893] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5426.179330] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5426.809354] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5427.445396] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5428.187337] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5428.914578] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5429.705100] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5430.501261] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5431.188434] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5431.868609] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5432.548841] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5433.338731] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5434.151161] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5434.920217] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5435.600481] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5436.290279] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5436.990458] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5437.694555] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5438.352994] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5439.051558] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5439.785150] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5440.502779] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5441.237384] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5441.861711] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5442.490546] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5443.069349] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5443.678193] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5444.301458] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5444.917467] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5445.590072] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5446.208040] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5446.770780] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5447.348457] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5447.916861] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5448.473145] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5449.021348] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5449.619488] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5450.191317] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5450.719939] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5451.295566] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5451.958677] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5452.543516] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5453.152223] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5453.713536] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5454.324489] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5454.904174] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5455.484890] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5456.047984] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5456.569990] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5457.134981] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5457.827232] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5458.386131] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5458.990884] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5459.570762] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5460.193348] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5460.751960] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5461.300716] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5461.812827] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5462.383596] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5462.955815] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5463.591468] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5464.148144] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5464.734039] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5465.275450] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5465.931884] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5466.473274] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5467.080398] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5467.691811] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5468.357192] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5468.924717] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5469.639395] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5470.282410] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5471.011540] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5471.707740] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5472.372179] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5473.034938] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5473.669412] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5474.208600] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5474.772504] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5475.320097] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5475.942063] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5476.502046] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5477.060106] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5477.676104] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5478.336067] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5478.994125] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5479.534554] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5480.099683] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5480.723991] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5481.325532] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5481.979283] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5482.632111] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5483.264495] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5483.888131] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5484.512859] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5485.161676] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5485.808126] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5486.386603] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5487.008915] LustreError: 162679:0:(osp_precreate.c:652:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5487.015357] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5487.016410] LustreError: 162679:0:(osp_precreate.c:1356:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5487.657551] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5488.378689] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5489.058570] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5489.721802] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5490.293421] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5490.945925] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5491.581323] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5492.258762] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5493.006940] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5493.714287] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5494.518740] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5576.674051] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 5576.682154] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5577.696769] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5577.703040] Lustre: Skipped 1 previous similar message [ 5581.709430] Lustre: server umount lustre-OST0000 complete [ 5584.353095] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 5584.361824] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5587.938015] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5587.942959] Lustre: Skipped 1 previous similar message [ 5590.349838] Lustre: server umount lustre-MDT0000 complete [ 5592.829126] LustreError: 162641:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1777499465 with bad export cookie 16445861908661175146 [ 5592.832598] LustreError: MGC192.168.203.158@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5592.843211] LustreError: 162641:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 5593.055416] Lustre: server umount lustre-MDT0001 complete [ 5599.945776] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing unload_modules_local [ 5601.834609] Key type lgssc unregistered [ 5602.032548] LNet: 217344:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5602.050861] LNetError: 217344:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5602.066130] LNet: Removed LNI 192.168.203.158@tcp [ 5602.594212] Key type .llcrypt unregistered [ 5602.596250] Key type ._llcrypt unregistered [ 5619.167365] Key type ._llcrypt registered [ 5619.169466] Key type .llcrypt registered [ 5619.280307] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing load_modules_local [ 5620.169638] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5620.188630] alg: No test for adler32 (adler32-zlib) [ 5621.177721] Lustre: Lustre: Build Version: 2.17.52_93_g6cdb58a [ 5621.394283] LNet: Added LNI 192.168.203.158@tcp [8/256/0/180] [ 5623.079321] Key type lgssc registered [ 5623.849768] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5629.829073] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5629.855733] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5631.162219] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5634.216392] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing set_default_debug -1 all [ 5639.898285] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5640.132715] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5642.841039] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing set_default_debug -1 all [ 5646.800023] Lustre: DEBUG MARKER: oleg358-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 5649.306384] Lustre: DEBUG MARKER: oleg358-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 5653.996783] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5654.291215] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5656.386984] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:12083 to 0x280000401:12129) [ 5659.220211] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing set_default_debug -1 all [ 5663.355962] Lustre: DEBUG MARKER: oleg358-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 5670.533761] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5671.764694] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5671.774173] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5671.779628] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5671.786874] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5671.787644] LustreError: 218740:0:(osp_precreate.c:1264:osp_precreate_thread()) cfs_fail_timeout id 2107 sleeping for 20000ms [ 5674.915339] Lustre: lustre-OST0000: Not available for connect from 192.168.203.58@tcp (stopping) [ 5676.000344] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 5676.009518] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5676.021676] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5677.024593] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5677.025767] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5679.267936] Lustre: server umount lustre-OST0000 complete [ 5691.807111] LustreError: 218740:0:(osp_precreate.c:1264:osp_precreate_thread()) cfs_fail_timeout id 2107 awake [ 5710.685602] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5710.919152] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5714.937386] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing set_default_debug -1 all [ 5717.095062] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5717.108790] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5718.293754] Lustre: DEBUG MARKER: oleg358-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 5719.315419] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid in FULL state after 0 sec [ 5721.678345] Lustre: DEBUG MARKER: oleg358-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff922b0610b000.ost_server_uuid 50 [ 5722.628549] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff922b0610b000.ost_server_uuid in FULL state after 0 sec [ 5727.717736] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5727.733275] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5727.746314] Lustre: Skipped 1 previous similar message [ 5731.168786] Lustre: server umount lustre-OST0000 complete [ 5733.858834] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5733.867302] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5733.875276] Lustre: Skipped 1 previous similar message [ 5733.880077] Lustre: Skipped 2 previous similar messages [ 5739.684917] Lustre: server umount lustre-MDT0000 complete [ 5741.876056] LustreError: 218702:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1777499614 with bad export cookie 2799647543408185753 [ 5741.879355] LustreError: MGC192.168.203.158@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5741.883291] LustreError: 218702:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 5742.114523] Lustre: server umount lustre-MDT0001 complete [ 5748.501327] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing unload_modules_local [ 5750.228743] Key type lgssc unregistered [ 5750.421059] LNet: 223354:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5750.426959] LNetError: 223354:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5750.440740] LNet: Removed LNI 192.168.203.158@tcp [ 5750.987227] Key type .llcrypt unregistered [ 5750.989106] Key type ._llcrypt unregistered [ 5769.026786] Key type ._llcrypt registered [ 5769.028432] Key type .llcrypt registered [ 5769.090947] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing load_modules_local [ 5769.734527] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5769.746708] alg: No test for adler32 (adler32-zlib) [ 5770.667812] Lustre: Lustre: Build Version: 2.17.52_93_g6cdb58a [ 5770.788447] LNet: Added LNI 192.168.203.158@tcp [8/256/0/180] [ 5772.407158] Key type lgssc registered [ 5773.117193] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5778.948937] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5778.963171] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5780.268242] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5783.241767] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing set_default_debug -1 all [ 5788.943785] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5789.153797] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5791.396706] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing set_default_debug -1 all [ 5793.216741] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 5797.138994] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5797.404949] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5801.311412] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing set_default_debug -1 all [ 5805.613133] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:12131 to 0x280000401:12161) [ 5807.914590] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5808.051544] Lustre: lustre-OST0001: new disk, initializing [ 5808.055415] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 5808.128756] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 5812.435446] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing set_default_debug -1 all [ 5815.375642] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 5815.382873] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 5815.475519] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 5821.535796] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 5824.503598] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 5835.233759] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 5835.242407] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5835.252943] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5836.260360] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5836.262321] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5836.272470] Lustre: Skipped 2 previous similar messages [ 5836.287254] Lustre: Skipped 2 previous similar messages [ 5838.823297] Lustre: server umount lustre-MDT0000 complete [ 5841.112718] LustreError: 226140:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1777499714 with bad export cookie 1585179812995056468 [ 5841.120827] LustreError: MGC192.168.203.158@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5841.122698] LustreError: 226140:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 5841.366796] Lustre: server umount lustre-MDT0001 complete [ 5853.143032] Lustre: server umount lustre-OST0000 complete [ 5865.813795] Lustre: server umount lustre-OST0001 complete [ 5878.315984] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing load_modules_local [ 5884.625413] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5884.996807] LustreError: 230374:0:(ldlm_lib.c:1180:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5885.006457] LustreError: 230374:0:(ldlm_lib.c:1180:target_handle_connect()) Skipped 2 previous similar messages [ 5885.054405] Lustre: mylustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5887.654397] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing set_default_debug -1 all [ 5890.530434] LustreError: 230374:0:(ldlm_lib.c:1180:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5890.555967] LustreError: 230374:0:(ldlm_lib.c:1180:target_handle_connect()) Skipped 1 previous similar message [ 5893.130580] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5893.425420] Lustre: mylustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5896.014768] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing set_default_debug -1 all [ 5897.958474] Lustre: 231481:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 5902.244858] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5903.532497] LustreError: 231835:0:(ldlm_lib.c:1180:target_handle_connect()) mylustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5906.396656] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing set_default_debug -1 all [ 5908.964749] LustreError: 231833:0:(ldlm_lib.c:1180:target_handle_connect()) mylustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5910.011598] Lustre: mylustre-OST0000: new connection from mylustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:12164 to 0x280000401:12193) [ 5912.360639] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5912.524723] Lustre: mylustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 5912.528437] Lustre: Skipped 1 previous similar message [ 5917.237842] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing set_default_debug -1 all [ 5923.389925] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 5925.719790] Lustre: 233317:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 5933.030382] Lustre: mylustre-MDT0000-lwp-OST0001: Connection to mylustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5933.034614] Lustre: mylustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5933.039150] Lustre: Skipped 3 previous similar messages [ 5933.041347] Lustre: Skipped 3 previous similar messages [ 5938.145888] Lustre: mylustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5938.148778] Lustre: Skipped 1 previous similar message [ 5943.277295] Lustre: mylustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5947.359267] Lustre: mylustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 5947.504762] Lustre: server umount mylustre-MDT0000 complete [ 5948.390597] LustreError: 230370:0:(ldlm_lib.c:1180:target_handle_connect()) mylustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5948.403987] LustreError: 230370:0:(ldlm_lib.c:1180:target_handle_connect()) Skipped 5 previous similar messages [ 5950.322210] LustreError: 230350:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1777499823 with bad export cookie 1585179812995059275 [ 5950.329115] LustreError: MGC192.168.203.158@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5950.332652] LustreError: 230350:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 5950.591583] Lustre: server umount mylustre-MDT0001 complete [ 5964.004484] Lustre: server umount mylustre-OST0000 complete [ 5977.108820] Lustre: server umount mylustre-OST0001 complete [ 5992.602203] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing load_modules_local [ 5998.733460] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5999.001290] LustreError: 235606:0:(ldlm_lib.c:1180:target_handle_connect()) tfs-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5999.058121] Lustre: tfs-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6001.599307] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing set_default_debug -1 all [ 6007.668976] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6010.791326] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing set_default_debug -1 all [ 6013.007860] Lustre: 236715:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 6017.408147] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6022.409392] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing set_default_debug -1 all [ 6023.913118] LustreError: 237069:0:(ldlm_lib.c:1180:target_handle_connect()) tfs-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6023.935037] Lustre: tfs-OST0000: new connection from tfs-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:12195 to 0x280000401:12225) [ 6023.940385] LustreError: 237069:0:(ldlm_lib.c:1180:target_handle_connect()) Skipped 1 previous similar message [ 6029.057132] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6033.466579] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing set_default_debug -1 all [ 6039.174377] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 6041.593625] Lustre: 238549:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 6047.716487] Lustre: tfs-MDT0000-lwp-OST0000: Connection to tfs-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6047.720764] Lustre: tfs-MDT0000: Not available for connect from 0@lo (stopping) [ 6047.729298] Lustre: Skipped 3 previous similar messages [ 6047.739680] Lustre: Skipped 6 previous similar messages [ 6052.186732] Lustre: server umount tfs-MDT0000 complete [ 6054.774415] LustreError: 235583:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1777499927 with bad export cookie 1585179812995060941 [ 6054.776355] LustreError: MGC192.168.203.158@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6054.785598] LustreError: 235583:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 6055.008774] Lustre: server umount tfs-MDT0001 complete [ 6067.255293] Lustre: server umount tfs-OST0000 complete [ 6079.614281] Lustre: server umount tfs-OST0001 complete [ 6090.546679] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing load_modules_local [ 6097.157075] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6097.488086] LustreError: 240778:0:(ldlm_lib.c:1180:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6097.503859] LustreError: 240778:0:(ldlm_lib.c:1180:target_handle_connect()) Skipped 4 previous similar messages [ 6097.555967] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6097.559269] Lustre: Skipped 3 previous similar messages [ 6100.241620] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing set_default_debug -1 all [ 6105.818562] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6108.835512] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing set_default_debug -1 all [ 6110.649721] Lustre: 241885:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 6114.867475] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6118.821900] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing set_default_debug -1 all [ 6122.485361] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:12195 to 0x280000401:12257) [ 6124.873790] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6129.022757] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing set_default_debug -1 all [ 6134.128890] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 6136.415495] Lustre: 243718:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 6140.890580] Lustre: DEBUG MARKER: == conf-sanity test 104a: Make sure user defined options are reflected in mount ========================================================== 18:00:12 (1777500012) [ 6155.748421] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6155.753135] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6155.756309] Lustre: Skipped 2 previous similar messages [ 6155.763704] Lustre: Skipped 3 previous similar messages [ 6160.582201] Lustre: server umount lustre-MDT0000 complete [ 6162.359214] LustreError: 240755:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1777500035 with bad export cookie 1585179812995062383 [ 6162.362946] LustreError: MGC192.168.203.158@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6162.363782] LustreError: 240755:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 6165.985456] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6165.986267] LustreError: 243980:0:(ldlm_lib.c:1180:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6165.994143] Lustre: Skipped 2 previous similar messages [ 6166.010132] LustreError: 243980:0:(ldlm_lib.c:1180:target_handle_connect()) Skipped 10 previous similar messages [ 6168.628767] Lustre: server umount lustre-MDT0001 complete [ 6176.607464] Lustre: server umount lustre-OST0000 complete [ 6184.625731] Lustre: server umount lustre-OST0001 complete [ 6187.848372] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing set_hostid [ 6193.220099] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing load_modules_local [ 6211.259396] LDISKFS-fs (dm-0): Mount option "noacl" will be removed by 3.5 [ 6211.259396] Contact linux-ldiskfs@vger.kernel.org if you think we should keep it. [ 6211.259396] [ 6211.282403] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: acl,user_xattr,errors=remount-ro,noacl,no_mbcache,nodelalloc [ 6211.441222] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 6211.470873] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 6211.530811] Lustre: lustre-MDT0000: new disk, initializing [ 6211.592436] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 6213.904693] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing set_default_debug -1 all [ 6219.845568] LDISKFS-fs (dm-1): Mount option "noacl" will be removed by 3.5 [ 6219.845568] Contact linux-ldiskfs@vger.kernel.org if you think we should keep it. [ 6219.845568] [ 6219.863308] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: acl,user_xattr,errors=remount-ro,noacl,no_mbcache,nodelalloc [ 6219.935576] Lustre: 246878:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 6219.981754] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 6219.985614] Lustre: Skipped 1 previous similar message [ 6220.035278] Lustre: lustre-MDT0001: new disk, initializing [ 6220.114439] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 6220.119276] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 6222.269447] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing set_default_debug -1 all [ 6228.777180] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6228.937925] Lustre: lustre-OST0000: new disk, initializing [ 6228.941394] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 6228.995971] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 6229.003437] Lustre: Skipped 5 previous similar messages [ 6230.888791] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 6230.898664] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 6230.962980] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 6232.241144] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing set_default_debug -1 all [ 6238.191454] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6238.284724] Lustre: lustre-OST0001: new disk, initializing [ 6238.288157] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 6240.087030] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 6240.094406] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 6240.138480] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 6241.659881] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing set_default_debug -1 all [ 6248.143740] Lustre: DEBUG MARKER: == conf-sanity test 104b: Mount uses last flock argument ========================================================== 18:01:59 (1777500119) [ 6254.310411] Lustre: DEBUG MARKER: == conf-sanity test 105: check file creation for ro and rw bind mnt pt ========================================================== 18:02:05 (1777500125) [ 6266.849598] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 6266.849736] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6266.855233] LustreError: Skipped 1 previous similar message [ 6266.856853] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6266.864262] Lustre: Skipped 1 previous similar message [ 6266.870874] Lustre: Skipped 4 previous similar messages [ 6281.183147] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 6281.250227] Lustre: server umount lustre-OST0000 complete [ 6289.606939] Lustre: server umount lustre-MDT0000 complete [ 6291.778653] LustreError: 246868:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1777500164 with bad export cookie 1585179812995063706 [ 6291.784730] LustreError: 246868:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 6291.785347] LustreError: MGC192.168.203.158@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6296.545146] LustreError: 246885:0:(ldlm_lib.c:1180:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6296.555029] LustreError: 246885:0:(ldlm_lib.c:1180:target_handle_connect()) Skipped 7 previous similar messages [ 6306.271323] Lustre: lustre-MDT0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 6306.391780] Lustre: server umount lustre-MDT0001 complete [ 6311.171537] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing unload_modules_local [ 6322.838343] Lustre: server umount lustre-OST0001 complete [ 6325.961295] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing set_hostid [ 6334.468785] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing load_modules_local [ 6353.509574] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6353.655568] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 6353.679542] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 6353.732899] Lustre: lustre-MDT0000: new disk, initializing [ 6353.798585] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 6355.932446] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing set_default_debug -1 all [ 6362.368939] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6362.446519] Lustre: 253740:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 6362.596867] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 6364.839373] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing set_default_debug -1 all [ 6369.283285] Lustre: DEBUG MARKER: oleg358-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 6371.270495] Lustre: DEBUG MARKER: oleg358-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 6374.894251] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6375.056920] Lustre: lustre-OST0000: new disk, initializing [ 6375.059143] Lustre: Skipped 1 previous similar message [ 6375.062374] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 6375.065538] Lustre: Skipped 2 previous similar messages [ 6376.859049] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 6376.864315] Lustre: Skipped 1 previous similar message [ 6376.869835] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 6376.930549] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 6378.075037] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing set_default_debug -1 all [ 6382.129789] Lustre: DEBUG MARKER: oleg358-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 6387.167955] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 6387.169970] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6387.173751] LustreError: Skipped 1 previous similar message [ 6387.183836] Lustre: Skipped 4 previous similar messages [ 6387.187615] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6387.197126] Lustre: Skipped 12 previous similar messages [ 6392.864843] Lustre: server umount lustre-OST0000 complete [ 6400.824586] Lustre: server umount lustre-MDT0000 complete [ 6402.870869] LustreError: 254517:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1777500275 with bad export cookie 1585179812995065176 [ 6402.874939] LustreError: MGC192.168.203.158@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6402.876805] LustreError: 254517:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 6403.044662] Lustre: server umount lustre-MDT0001 complete [ 6408.946734] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing unload_modules_local [ 6410.417098] Key type lgssc unregistered [ 6410.578901] LNet: 257106:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 6410.584197] LNetError: 257106:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 6410.596901] LNet: Removed LNI 192.168.203.158@tcp [ 6411.005151] Key type .llcrypt unregistered [ 6411.012113] Key type ._llcrypt unregistered [ 6424.954213] Key type ._llcrypt registered [ 6424.955915] Key type .llcrypt registered [ 6425.023631] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing load_modules_local [ 6425.598928] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 6425.606837] alg: No test for adler32 (adler32-zlib) [ 6426.545584] Lustre: Lustre: Build Version: 2.17.52_93_g6cdb58a [ 6426.677666] LNet: Added LNI 192.168.203.158@tcp [8/256/0/180] [ 6428.311772] Key type lgssc registered [ 6428.909641] Lustre: Echo OBD driver; http://www.lustre.org/ [ 6433.018862] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 6433.033514] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6434.318057] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6436.456954] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing set_default_debug -1 all [ 6441.086770] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6441.290733] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 6443.361158] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing set_default_debug -1 all [ 6446.042454] Lustre: DEBUG MARKER: oleg358-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 6447.777806] Lustre: DEBUG MARKER: oleg358-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 6451.153635] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6451.315583] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 6454.071197] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing set_default_debug -1 all [ 6456.817554] Lustre: DEBUG MARKER: oleg358-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 6457.390403] Lustre: Setting parameter lustre-OST0000.ost.unknown_param=50 in log lustre-OST0000 [ 6459.425684] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6459.429969] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6464.498759] Lustre: server umount lustre-OST0000 complete [ 6467.041567] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 6467.047618] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6467.058252] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6467.061714] Lustre: Skipped 1 previous similar message [ 6469.600332] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6469.601231] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6472.708773] Lustre: server umount lustre-MDT0000 complete [ 6474.682640] LustreError: 260719:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1777500347 with bad export cookie 13125172294842992099 [ 6474.684631] LustreError: MGC192.168.203.158@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6474.691233] LustreError: 260719:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 6474.872466] Lustre: server umount lustre-MDT0001 complete [ 6480.046178] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing unload_modules_local [ 6481.538996] Key type lgssc unregistered [ 6481.738254] LNet: 261762:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 6481.741693] LNetError: 261762:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 6481.758297] LNet: Removed LNI 192.168.203.158@tcp [ 6482.203666] Key type .llcrypt unregistered [ 6482.206754] Key type ._llcrypt unregistered [ 6491.940145] Key type ._llcrypt registered [ 6491.941772] Key type .llcrypt registered [ 6492.003946] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing load_modules_local [ 6492.444208] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 6492.452588] alg: No test for adler32 (adler32-zlib) [ 6493.407543] Lustre: Lustre: Build Version: 2.17.52_93_g6cdb58a [ 6493.521137] LNet: Added LNI 192.168.203.158@tcp [8/256/0/180] [ 6495.135241] Key type lgssc registered [ 6495.763104] Lustre: Echo OBD driver; http://www.lustre.org/ [ 6500.300947] Lustre: lustre-OST0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 6500.317475] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6515.808079] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 6520.927785] LustreError: 262918:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.203.158@tcp: failed processing log, type 4: rc = -110 [ 6547.551653] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 6550.551940] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing set_default_debug -1 all [ 6553.163821] Lustre: DEBUG MARKER: oleg358-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 6554.288502] Lustre: server umount lustre-OST0000 complete [ 6561.020107] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing unload_modules_local [ 6562.168602] Key type lgssc unregistered [ 6562.306490] LNet: 264416:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 6562.309917] LNetError: 264416:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 6562.318433] LNet: Removed LNI 192.168.203.158@tcp [ 6562.650189] Key type .llcrypt unregistered [ 6562.652424] Key type ._llcrypt unregistered [ 6579.144147] Key type ._llcrypt registered [ 6579.145566] Key type .llcrypt registered [ 6579.196292] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing load_modules_local [ 6579.600731] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 6579.655263] alg: No test for adler32 (adler32-zlib) [ 6580.540317] Lustre: Lustre: Build Version: 2.17.52_93_g6cdb58a [ 6580.646450] LNet: Added LNI 192.168.203.158@tcp [8/256/0/180] [ 6582.231155] Key type lgssc registered [ 6582.654935] Lustre: Echo OBD driver; http://www.lustre.org/ [ 6587.699960] loop0: detected capacity change from 0 to 204800000 [ 6587.768242] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 6587.773319] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 6587.917809] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 6588.353385] loop0: detected capacity change from 0 to 204800000 [ 6588.445829] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 6588.451151] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 6588.616993] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 6589.052151] loop0: detected capacity change from 0 to 204800000 [ 6589.167793] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 6589.173293] blk_update_request: operation not supported error, dev loop0, sector 8224 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 6589.228703] blk_update_request: operation not supported error, dev loop0, sector 58264 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 6589.748354] loop0: detected capacity change from 0 to 204800000 [ 6589.818437] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 6590.333333] loop0: detected capacity change from 0 to 204800000 [ 6590.350471] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 6592.053370] loop0: detected capacity change from 0 to 204800000 [ 6592.064962] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 6593.811722] loop0: detected capacity change from 0 to 204800000 [ 6593.824084] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 6595.426073] loop0: detected capacity change from 0 to 204800000 [ 6595.439972] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 6596.890161] loop0: detected capacity change from 0 to 204800000 [ 6596.921572] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 6596.931551] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6596.962986] Lustre: 267788:0:(osd_scrub.c:1830:osd_index_restore()) lustre-MDT0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 6596.967660] Lustre: 267788:0:(osd_scrub.c:1830:osd_index_restore()) lustre-MDT0000: restore index 'nodemap' with [0x200000003:0x3:0x0]: rc = 0 [ 6596.972160] Lustre: 267788:0:(osd_scrub.c:1830:osd_index_restore()) lustre-MDT0000: restore index '0x2010000-MDT0000' with [0x200000005:0x3:0x0]: rc = 0 [ 6596.977926] Lustre: 267788:0:(osd_scrub.c:1830:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000003:0xc:0x0]: rc = 0 [ 6596.983801] Lustre: 267788:0:(osd_scrub.c:1830:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000003:0xa:0x0]: rc = 0 [ 6596.990138] Lustre: 267788:0:(osd_scrub.c:1830:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000003:0xb:0x0]: rc = 0 [ 6596.995937] Lustre: 267788:0:(osd_scrub.c:1830:osd_index_restore()) lustre-MDT0000: restore index '0x1010000-MDT0000' with [0x200000005:0x2:0x0]: rc = 0 [ 6597.002269] Lustre: 267788:0:(osd_scrub.c:1830:osd_index_restore()) lustre-MDT0000: restore index '0x10000-MDT0000' with [0x200000005:0x1:0x0]: rc = 0 [ 6597.007987] Lustre: 267788:0:(osd_scrub.c:1830:osd_index_restore()) lustre-MDT0000: restore index '0x2020000' with [0x200000006:0x2020000:0x0]: rc = 0 [ 6597.013537] Lustre: 267788:0:(osd_scrub.c:1830:osd_index_restore()) lustre-MDT0000: restore index '0x20000' with [0x200000006:0x20000:0x0]: rc = 0 [ 6597.018684] Lustre: 267788:0:(osd_scrub.c:1830:osd_index_restore()) lustre-MDT0000: restore index '0x1020000' with [0x200000006:0x1020000:0x0]: rc = 0 [ 6597.023192] Lustre: 267788:0:(osd_scrub.c:1830:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000006:0x2010000:0x0]: rc = 0 [ 6597.028303] Lustre: 267788:0:(osd_scrub.c:1830:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000006:0x10000:0x0]: rc = 0 [ 6597.033404] Lustre: 267788:0:(osd_scrub.c:1830:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000006:0x1010000:0x0]: rc = 0 [ 6599.888939] Lustre: server umount MGS complete [ 6600.206156] loop0: detected capacity change from 0 to 204800000 [ 6600.236241] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6600.412780] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6600.430957] Lustre: lustre-MDT0000: Aborting client recovery [ 6600.763734] loop1: detected capacity change from 0 to 204800000 [ 6600.795075] LDISKFS-fs (loop1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6600.834743] Lustre: 268221:0:(osd_scrub.c:1830:osd_index_restore()) lustre-MDT0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 6600.840217] Lustre: 268221:0:(osd_scrub.c:1830:osd_index_restore()) lustre-MDT0001: restore index '0x10000' with [0x200000003:0x4:0x0]: rc = 0 [ 6600.845167] Lustre: 268221:0:(osd_scrub.c:1830:osd_index_restore()) lustre-MDT0001: restore index '0x10000-MDT0001' with [0x200000005:0x1:0x0]: rc = 0 [ 6600.850438] Lustre: 268221:0:(osd_scrub.c:1830:osd_index_restore()) lustre-MDT0001: restore index '0x2010000' with [0x200000003:0x6:0x0]: rc = 0 [ 6600.855796] Lustre: 268221:0:(osd_scrub.c:1830:osd_index_restore()) lustre-MDT0001: restore index '0x2010000-MDT0001' with [0x200000005:0x3:0x0]: rc = 0 [ 6600.861349] Lustre: 268221:0:(osd_scrub.c:1830:osd_index_restore()) lustre-MDT0001: restore index '0x1010000' with [0x200000003:0x5:0x0]: rc = 0 [ 6600.866853] Lustre: 268221:0:(osd_scrub.c:1830:osd_index_restore()) lustre-MDT0001: restore index '0x1010000-MDT0001' with [0x200000005:0x2:0x0]: rc = 0 [ 6600.877464] Lustre: Found index 1 for lustre-MDT0001, updating log [ 6600.953639] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 6600.984192] Lustre: lustre-MDT0001: Aborting client recovery [ 6601.350871] loop2: detected capacity change from 0 to 204800000 [ 6601.385760] LDISKFS-fs (loop2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6601.433345] Lustre: 268300:0:(osd_scrub.c:1830:osd_index_restore()) lustre-OST0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 6601.439730] Lustre: 268300:0:(osd_scrub.c:1830:osd_index_restore()) lustre-OST0000: restore index '0x20000-OST0000' with [0x200000005:0x1:0x0]: rc = 0 [ 6601.447632] Lustre: 268300:0:(osd_scrub.c:1830:osd_index_restore()) lustre-OST0000: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 6601.454408] Lustre: 268300:0:(osd_scrub.c:1830:osd_index_restore()) lustre-OST0000: restore index '0x2020000-OST0000' with [0x200000005:0x3:0x0]: rc = 0 [ 6601.460167] Lustre: 268300:0:(osd_scrub.c:1830:osd_index_restore()) lustre-OST0000: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 6601.472136] Lustre: 268300:0:(osd_scrub.c:1830:osd_index_restore()) lustre-OST0000: restore index '0x1020000-OST0000' with [0x200000005:0x2:0x0]: rc = 0 [ 6601.477176] Lustre: 268300:0:(osd_scrub.c:1830:osd_index_restore()) lustre-OST0000: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 6601.534277] Lustre: Found index 0 for lustre-OST0000, updating log [ 6601.577082] Lustre: lustre-OST0000: Aborting recovery [ 6601.967297] loop3: detected capacity change from 0 to 204800000 [ 6602.007631] LDISKFS-fs (loop3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6602.050745] Lustre: 268385:0:(osd_scrub.c:1830:osd_index_restore()) lustre-OST0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 6602.056161] Lustre: 268385:0:(osd_scrub.c:1830:osd_index_restore()) lustre-OST0001: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 6602.062133] Lustre: 268385:0:(osd_scrub.c:1830:osd_index_restore()) lustre-OST0001: restore index '0x2020000-OST0001' with [0x200000005:0x3:0x0]: rc = 0 [ 6602.068463] Lustre: 268385:0:(osd_scrub.c:1830:osd_index_restore()) lustre-OST0001: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 6602.074469] Lustre: 268385:0:(osd_scrub.c:1830:osd_index_restore()) lustre-OST0001: restore index '0x1020000-OST0001' with [0x200000005:0x2:0x0]: rc = 0 [ 6602.080104] Lustre: 268385:0:(osd_scrub.c:1830:osd_index_restore()) lustre-OST0001: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 6602.084857] Lustre: 268385:0:(osd_scrub.c:1830:osd_index_restore()) lustre-OST0001: restore index '0x20000-OST0001' with [0x200000005:0x1:0x0]: rc = 0 [ 6602.128885] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 6602.131662] Lustre: Skipped 1 previous similar message [ 6602.140950] Lustre: lustre-OST0001: Aborting recovery [ 6607.343944] LustreError: 268165:0:(lod_dev.c:510:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 7, retries 0, failed: rc = -108 [ 6610.401915] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 6610.403440] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:5 to 0x280000400:33) [ 6610.409578] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 6610.413298] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000bd0 [ 6610.418171] Lustre: lustre-OST0001: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:5 to 0x2c0000400:33) [ 6610.432143] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000bd0 [ 6612.868627] LustreError: 268131:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0000: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0000 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0000/identity_upcall, time 189us: rc = -2 [ 6625.760538] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6625.762108] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6625.766880] Lustre: Skipped 2 previous similar messages [ 6625.769832] Lustre: Skipped 1 previous similar message [ 6630.380130] Lustre: server umount lustre-MDT0000 complete [ 6630.725899] LustreError: 268115:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1777500503 with bad export cookie 13036738675536434342 [ 6630.728920] LustreError: MGC192.168.203.158@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6630.731244] LustreError: 268115:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 6630.879807] LustreError: 268399:0:(ldlm_lib.c:1180:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6630.880146] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6630.880412] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 6630.880419] Lustre: Skipped 2 previous similar messages [ 6630.887914] LustreError: 268399:0:(ldlm_lib.c:1180:target_handle_connect()) Skipped 3 previous similar messages [ 6630.892362] Lustre: Skipped 2 previous similar messages [ 6635.999457] LustreError: 268128:0:(ldlm_lib.c:1180:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6635.999560] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 6636.007352] LustreError: 268128:0:(ldlm_lib.c:1180:target_handle_connect()) Skipped 1 previous similar message [ 6636.010598] Lustre: Skipped 2 previous similar messages [ 6640.096122] LustreError: 268617:0:(ldlm_lib.c:1180:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6640.096153] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 6640.103482] LustreError: 268617:0:(ldlm_lib.c:1180:target_handle_connect()) Skipped 1 previous similar message [ 6640.107116] Lustre: Skipped 1 previous similar message [ 6645.215114] Lustre: lustre-MDT0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 6645.215720] LustreError: 268130:0:(ldlm_lib.c:1180:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6645.215806] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 6645.229668] LustreError: 268130:0:(ldlm_lib.c:1180:target_handle_connect()) Skipped 1 previous similar message [ 6645.341563] Lustre: server umount lustre-MDT0001 complete [ 6651.784172] Lustre: server umount lustre-OST0000 complete [ 6658.276763] Lustre: server umount lustre-OST0001 complete [ 6661.420938] Lustre: DEBUG MARKER: == conf-sanity test 109a: test lctl clear_conf fsname ==== 18:08:53 (1777500533) [ 6672.935454] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing set_hostid [ 6677.002539] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing load_modules_local [ 6692.439262] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6692.547995] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 6692.576171] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 6692.628243] Lustre: lustre-MDT0000: new disk, initializing [ 6692.671080] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6692.679990] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 6694.543517] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing set_default_debug -1 all [ 6699.794726] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6699.845983] Lustre: 272417:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 6699.882990] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 6699.886187] Lustre: Skipped 1 previous similar message [ 6699.926922] Lustre: lustre-MDT0001: new disk, initializing [ 6699.978294] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 6699.996048] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 6700.002915] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 6701.851929] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing set_default_debug -1 all [ 6705.603807] Lustre: DEBUG MARKER: oleg358-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 6707.153895] Lustre: DEBUG MARKER: oleg358-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 6709.887696] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6710.006382] Lustre: lustre-OST0000: new disk, initializing [ 6710.008952] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 6710.048700] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 6712.248785] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing set_default_debug -1 all [ 6715.793804] Lustre: DEBUG MARKER: oleg358-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 6717.950621] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 6717.955623] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 6718.001584] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 6723.973919] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [ 6726.927524] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [ 6734.656069] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [ 6772.194226] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6772.195447] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6772.200585] Lustre: Skipped 1 previous similar message [ 6772.204544] Lustre: Skipped 2 previous similar messages [ 6777.686680] Lustre: server umount lustre-OST0000 complete [ 6781.921775] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 6781.924937] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6785.366765] Lustre: server umount lustre-MDT0000 complete [ 6786.747679] LustreError: 275149:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1777500659 with bad export cookie 13036738675536437044 [ 6786.750059] LustreError: MGC192.168.203.158@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6786.754680] LustreError: 275149:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 6786.903822] Lustre: server umount lustre-MDT0001 complete [ 6790.367367] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6791.828924] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing set_default_debug -1 all [ 6793.735655] Lustre: server umount MGS complete [ 6798.885487] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6799.109705] LustreError: 277888:0:(ldlm_lib.c:1180:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6799.120881] LustreError: 277901:0:(qmt_pool.c:1630:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [ 6799.150675] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6800.779946] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing set_default_debug -1 all [ 6804.666155] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6806.620537] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing set_default_debug -1 all [ 6808.889552] Lustre: DEBUG MARKER: oleg358-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 6810.298369] Lustre: DEBUG MARKER: oleg358-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 6813.043823] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6815.521118] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing set_default_debug -1 all [ 6817.711707] Lustre: DEBUG MARKER: oleg358-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 6844.896352] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 6844.899020] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6844.900562] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6844.900567] Lustre: Skipped 5 previous similar messages [ 6844.905234] LustreError: Skipped 1 previous similar message [ 6844.912929] Lustre: Skipped 2 previous similar messages [ 6849.176320] Lustre: server umount lustre-OST0000 complete [ 6851.040156] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 6855.137901] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6855.143576] Lustre: Skipped 1 previous similar message [ 6865.375152] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 6865.457910] Lustre: server umount lustre-MDT0000 complete [ 6866.935455] LustreError: 277868:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1777500739 with bad export cookie 13036738675536438857 [ 6866.938635] LustreError: MGC192.168.203.158@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6866.940697] LustreError: 277868:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 6867.069593] Lustre: server umount lustre-MDT0001 complete [ 6871.313143] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing unload_modules_local [ 6872.475892] Key type lgssc unregistered [ 6872.606355] LNet: 281497:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 6872.609758] LNetError: 281497:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 6872.618293] LNet: Removed LNI 192.168.203.158@tcp [ 6872.934982] Key type .llcrypt unregistered [ 6872.936603] Key type ._llcrypt unregistered [ 6886.897035] Key type ._llcrypt registered [ 6886.898439] Key type .llcrypt registered [ 6886.942200] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing set_hostid [ 6893.644783] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing load_modules_local [ 6894.021763] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 6894.114335] alg: No test for adler32 (adler32-zlib) [ 6894.990055] Lustre: Lustre: Build Version: 2.17.52_93_g6cdb58a [ 6895.078581] LNet: Added LNI 192.168.203.158@tcp [8/256/0/180] [ 6896.671161] Key type lgssc registered [ 6897.132467] Lustre: Echo OBD driver; http://www.lustre.org/ [ 6910.755445] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 6910.763213] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6911.877096] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 6911.891555] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 6911.931236] Lustre: lustre-MDT0000: new disk, initializing [ 6911.965618] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6911.974747] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 6913.432318] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing set_default_debug -1 all [ 6917.893183] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6917.942232] Lustre: 285390:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 6917.968261] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 6917.970967] Lustre: Skipped 1 previous similar message [ 6918.010306] Lustre: lustre-MDT0001: new disk, initializing [ 6918.054213] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 6918.069949] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 6918.076499] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 6919.533936] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing set_default_debug -1 all [ 6922.713490] Lustre: DEBUG MARKER: oleg358-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 6923.975328] Lustre: DEBUG MARKER: oleg358-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 6926.309091] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6926.404053] Lustre: lustre-OST0000: new disk, initializing [ 6926.405610] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 6926.439165] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 6926.937179] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 6926.941249] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 6926.990922] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 6928.349629] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing set_default_debug -1 all [ 6931.770532] Lustre: DEBUG MARKER: oleg358-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 6935.436270] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [ 6946.072726] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [ 6952.075442] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [ 6999.010602] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6999.011531] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 6999.012199] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6999.016956] Lustre: Skipped 1 previous similar message [ 7004.129685] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7004.132693] Lustre: Skipped 2 previous similar messages [ 7009.250118] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7009.252797] Lustre: Skipped 1 previous similar message [ 7011.295130] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 7011.349477] Lustre: server umount lustre-OST0000 complete [ 7014.369108] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7014.371180] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7014.374596] Lustre: Skipped 1 previous similar message [ 7018.839728] Lustre: server umount lustre-MDT0000 complete [ 7020.166598] LustreError: 285384:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1777500893 with bad export cookie 9710201619125833668 [ 7020.170637] LustreError: MGC192.168.203.158@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7020.171070] LustreError: 285384:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 7020.287556] Lustre: server umount lustre-MDT0001 complete [ 7023.268400] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7024.814792] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing set_default_debug -1 all [ 7026.680512] Lustre: server umount MGS complete [ 7031.467117] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7031.700245] LustreError: 291104:0:(ldlm_lib.c:1180:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7031.706641] LustreError: 291104:0:(ldlm_lib.c:1180:target_handle_connect()) Skipped 3 previous similar messages [ 7031.713431] LustreError: 291117:0:(qmt_pool.c:1630:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [ 7031.745749] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7033.374340] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing set_default_debug -1 all [ 7036.726747] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7036.915966] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7038.524963] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing set_default_debug -1 all [ 7040.697854] Lustre: DEBUG MARKER: oleg358-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 7042.170064] Lustre: DEBUG MARKER: oleg358-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 7044.681291] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7046.762981] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing set_default_debug -1 all [ 7048.786513] Lustre: DEBUG MARKER: oleg358-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 7081.952461] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7081.953511] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 7081.953986] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7081.953989] Lustre: Skipped 1 previous similar message [ 7081.956100] Lustre: Skipped 1 previous similar message [ 7081.958143] LustreError: Skipped 1 previous similar message [ 7087.313679] Lustre: server umount lustre-OST0000 complete [ 7092.192523] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7092.193211] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7092.197760] Lustre: Skipped 1 previous similar message [ 7092.199761] Lustre: Skipped 3 previous similar messages [ 7094.687419] Lustre: server umount lustre-MDT0000 complete [ 7096.060219] LustreError: 291086:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1777500968 with bad export cookie 9710201619125835453 [ 7096.062508] LustreError: MGC192.168.203.158@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7096.065804] LustreError: 291086:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 7096.212883] Lustre: server umount lustre-MDT0001 complete [ 7100.320545] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing unload_modules_local [ 7101.460664] Key type lgssc unregistered [ 7101.600379] LNet: 294712:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 7101.603810] LNetError: 294712:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 7101.613415] LNet: Removed LNI 192.168.203.158@tcp [ 7101.946229] Key type .llcrypt unregistered [ 7101.947600] Key type ._llcrypt unregistered [ 7109.014821] Key type ._llcrypt registered [ 7109.016207] Key type .llcrypt registered [ 7109.058928] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing set_hostid [ 7115.487538] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing load_modules_local [ 7115.844197] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 7115.894054] alg: No test for adler32 (adler32-zlib) [ 7116.786225] Lustre: Lustre: Build Version: 2.17.52_93_g6cdb58a [ 7116.888308] LNet: Added LNI 192.168.203.158@tcp [8/256/0/180] [ 7118.487211] Key type lgssc registered [ 7118.910653] Lustre: Echo OBD driver; http://www.lustre.org/ [ 7127.842286] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing load_modules_local [ 7133.921710] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 7133.928519] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7135.024495] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 7135.035875] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 7135.074365] Lustre: lustre-MDT0000: new disk, initializing [ 7135.106880] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7135.115393] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 7136.634671] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing set_default_debug -1 all [ 7141.221033] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7141.336581] Lustre: lustre-OST0000: new disk, initializing [ 7141.339443] Lustre: lustre-OST0000: Not available for connect from 0@lo (not set up) [ 7141.339567] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 7141.345725] Lustre: Skipped 1 previous similar message [ 7141.371758] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7143.320568] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing set_default_debug -1 all [ 7146.477515] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 7146.482060] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 7146.493670] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 7380.658491] Lustre: server umount lustre-MDT0000 complete [ 7382.187421] LustreError: 297870:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1777501255 with bad export cookie 12079558317627414902 [ 7382.192414] LustreError: MGC192.168.203.158@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7392.804298] Lustre: server umount lustre-OST0000 complete [ 7402.581297] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing set_hostid [ 7406.308559] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing load_modules_local [ 7419.878721] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7419.973842] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 7419.986263] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 7420.024245] Lustre: lustre-MDT0000: new disk, initializing [ 7420.051259] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7420.059524] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 7421.405721] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing set_default_debug -1 all [ 7425.799707] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7425.847218] Lustre: 302850:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 7425.872606] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 7425.875552] Lustre: Skipped 1 previous similar message [ 7425.911664] Lustre: lustre-MDT0001: new disk, initializing [ 7425.953856] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7425.966514] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 7425.971922] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 7427.332902] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing set_default_debug -1 all [ 7430.521405] Lustre: DEBUG MARKER: oleg358-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 7431.778675] Lustre: DEBUG MARKER: oleg358-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 7434.091740] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7434.217744] Lustre: lustre-OST0000: new disk, initializing [ 7434.220661] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 7434.249187] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7436.215489] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing set_default_debug -1 all [ 7439.345850] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 7439.350136] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 7439.361869] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000400 [ 7439.382340] Lustre: DEBUG MARKER: oleg358-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 7440.841026] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 7440.915340] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 7442.243230] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 7443.336350] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 1 sec [ 7444.448041] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 7444.450983] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7444.455264] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7447.524065] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7447.524323] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 7447.530279] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7450.071186] Lustre: server umount lustre-OST0000 complete [ 7451.615839] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7451.618714] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7451.623278] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7451.625595] Lustre: Skipped 1 previous similar message [ 7457.563793] Lustre: server umount lustre-MDT0000 complete [ 7457.759742] LustreError: 302862:0:(ldlm_lib.c:1180:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7457.766099] LustreError: 302862:0:(ldlm_lib.c:1180:target_handle_connect()) Skipped 3 previous similar messages [ 7458.850071] LustreError: 303627:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1777501331 with bad export cookie 12079558317630355700 [ 7458.853730] LustreError: MGC192.168.203.158@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7458.854609] LustreError: 303627:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 7458.973694] Lustre: server umount lustre-MDT0001 complete [ 7460.128323] Lustre: DEBUG MARKER: SKIP: conf-sanity test_111 skipping SLOW test 111 [ 7460.698082] Lustre: DEBUG MARKER: == conf-sanity test 112a: mount OST with no_create option ========================================================== 18:22:12 (1777501332) [ 7462.860035] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7463.006236] LustreError: 306763:0:(ldlm_lib.c:1180:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7463.028995] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7464.308273] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing set_default_debug -1 all [ 7467.252391] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7468.737685] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing set_default_debug -1 all [ 7470.550246] Lustre: DEBUG MARKER: oleg358-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 7471.807870] Lustre: DEBUG MARKER: oleg358-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 7474.212266] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7476.537079] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing set_default_debug -1 all [ 7478.737140] Lustre: DEBUG MARKER: oleg358-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 7481.422622] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7481.496384] Lustre: lustre-OST0001: new disk, initializing [ 7481.498941] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 7481.543658] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 7481.546707] Lustre: Skipped 2 previous similar messages [ 7483.884066] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing set_default_debug -1 all [ 7488.513727] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 7488.518495] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 7488.532408] LustreError: 309812:0:(osp_precreate.c:970:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [ 7488.536211] LustreError: 309812:0:(osp_precreate.c:970:osp_precreate_cleanup_orphans()) Skipped 1 previous similar message [ 7489.569651] LustreError: 309817:0:(osp_precreate.c:970:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [ 7489.572877] LustreError: 309817:0:(osp_precreate.c:970:osp_precreate_cleanup_orphans()) Skipped 1 previous similar message [ 7490.594320] LustreError: 309817:0:(osp_precreate.c:970:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [ 7490.599056] LustreError: 309817:0:(osp_precreate.c:970:osp_precreate_cleanup_orphans()) Skipped 1 previous similar message [ 7492.642271] LustreError: 309812:0:(osp_precreate.c:970:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [ 7492.645579] LustreError: 309812:0:(osp_precreate.c:970:osp_precreate_cleanup_orphans()) Skipped 3 previous similar messages [ 7492.784803] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 7492.855158] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 7494.108169] Lustre: DEBUG MARKER: oleg358-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff922b03108800.ost_server_uuid 50 [ 7494.617487] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff922b03108800.ost_server_uuid in FULL state after 0 sec [ 7495.798931] Lustre: DEBUG MARKER: oleg358-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff922b03108800.ost_server_uuid 50 [ 7496.319406] Lustre: DEBUG MARKER: osc.lustre-OST0001-osc-ffff922b03108800.ost_server_uuid in FULL state after 0 sec [ 7496.738220] LustreError: 309817:0:(osp_precreate.c:970:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [ 7496.741773] LustreError: 309817:0:(osp_precreate.c:970:osp_precreate_cleanup_orphans()) Skipped 7 previous similar messages [ 7497.770226] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 7507.936027] LustreError: lustre-OST0001-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 7507.936081] Lustre: lustre-OST0001-osc-MDT0001: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7507.938571] LustreError: Skipped 1 previous similar message [ 7507.943583] Lustre: Skipped 1 previous similar message [ 7507.946502] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 7507.949056] Lustre: Skipped 3 previous similar messages [ 7517.069279] Lustre: lustre-OST0001: Not available for connect from 192.168.203.58@tcp (stopping) [ 7517.071772] Lustre: Skipped 5 previous similar messages [ 7522.271093] Lustre: lustre-OST0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 7522.305305] Lustre: server umount lustre-OST0001 complete [ 7524.321652] LustreError: 308355:0:(ldlm_lib.c:1180:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7524.322323] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7524.326684] LustreError: 308355:0:(ldlm_lib.c:1180:target_handle_connect()) Skipped 1 previous similar message [ 7524.330017] Lustre: Skipped 1 previous similar message [ 7529.443653] LustreError: 308356:0:(ldlm_lib.c:1180:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7529.450581] LustreError: 308356:0:(ldlm_lib.c:1180:target_handle_connect()) Skipped 1 previous similar message [ 7530.002299] Lustre: server umount lustre-OST0000 complete [ 7534.048147] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7534.051535] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7534.057317] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7534.060049] Lustre: Skipped 7 previous similar messages [ 7537.561604] Lustre: server umount lustre-MDT0000 complete [ 7538.864050] LustreError: 306744:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1777501411 with bad export cookie 12079558317630356568 [ 7538.867606] LustreError: MGC192.168.203.158@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7538.869344] LustreError: 306744:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 7538.987539] Lustre: server umount lustre-MDT0001 complete [ 7542.586526] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing unload_modules_local [ 7543.608570] Key type lgssc unregistered [ 7543.746380] LNet: 311681:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 7543.749855] LNetError: 311681:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 7543.757818] LNet: Removed LNI 192.168.203.158@tcp [ 7544.057561] Key type .llcrypt unregistered [ 7544.058805] Key type ._llcrypt unregistered [ 7553.505299] Key type ._llcrypt registered [ 7553.506363] Key type .llcrypt registered [ 7553.545638] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing load_modules_local [ 7553.954042] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 7553.965647] alg: No test for adler32 (adler32-zlib) [ 7554.813234] Lustre: Lustre: Build Version: 2.17.52_93_g6cdb58a [ 7554.895334] LNet: Added LNI 192.168.203.158@tcp [8/256/0/180] [ 7556.479097] Key type lgssc registered [ 7556.840770] Lustre: Echo OBD driver; http://www.lustre.org/ [ 7559.501129] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 7559.507269] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7560.663694] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7561.958388] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing set_default_debug -1 all [ 7563.758470] Lustre: DEBUG MARKER: oleg358-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 7566.111858] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7566.238183] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7567.595790] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing set_default_debug -1 all [ 7570.830255] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7570.947663] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7572.807618] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing set_default_debug -1 all [ 7574.545192] Lustre: DEBUG MARKER: oleg358-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 7576.096893] LustreError: 314605:0:(ldlm_lib.c:1180:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7576.105175] LustreError: 314605:0:(ldlm_lib.c:1180:target_handle_connect()) Skipped 2 previous similar messages [ 7576.802943] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7576.869785] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 7578.692392] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing set_default_debug -1 all [ 7580.386782] Lustre: DEBUG MARKER: oleg358-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid [ 7585.257115] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:33) [ 7585.260297] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000400:36 to 0x280000400:65) [ 7587.089916] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing wait_import_state (FULL|IDLE) os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [ 7587.163824] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 7664.096029] LustreError: lustre-OST0001-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 7664.099950] Lustre: lustre-OST0001-osc-MDT0001: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7664.105301] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 7667.169655] LustreError: lustre-OST0001-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 7667.170422] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 7667.174096] Lustre: lustre-OST0001-osc-MDT0000: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7668.602331] Lustre: server umount lustre-OST0001 complete [ 7672.288668] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 7672.290361] LustreError: 316363:0:(ldlm_lib.c:1180:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7672.291152] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7672.292466] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7672.292470] Lustre: Skipped 1 previous similar message [ 7672.296414] LustreError: 316363:0:(ldlm_lib.c:1180:target_handle_connect()) Skipped 1 previous similar message [ 7672.301446] Lustre: Skipped 1 previous similar message [ 7676.240818] Lustre: server umount lustre-OST0000 complete [ 7678.944027] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7678.947178] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7678.952084] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7678.955326] Lustre: Skipped 1 previous similar message [ 7683.673905] Lustre: server umount lustre-MDT0000 complete [ 7684.909345] LustreError: 313041:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1777501557 with bad export cookie 9888373177668265209 [ 7684.914175] LustreError: MGC192.168.203.158@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7684.916726] LustreError: 313041:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 7685.036029] Lustre: server umount lustre-MDT0001 complete [ 7688.642975] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing unload_modules_local [ 7689.695848] Key type lgssc unregistered [ 7689.827637] LNet: 317602:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 7689.830218] LNetError: 317602:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 7689.837338] LNet: Removed LNI 192.168.203.158@tcp [ 7690.135581] Key type .llcrypt unregistered [ 7690.137127] Key type ._llcrypt unregistered [ 7707.623267] Key type ._llcrypt registered [ 7707.624262] Key type .llcrypt registered [ 7707.663361] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing load_modules_local [ 7707.924161] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 7707.942116] alg: No test for adler32 (adler32-zlib) [ 7708.818534] Lustre: Lustre: Build Version: 2.17.52_93_g6cdb58a [ 7708.916850] LNet: Added LNI 192.168.203.158@tcp [8/256/0/180] [ 7710.503150] Key type lgssc registered [ 7710.874461] Lustre: Echo OBD driver; http://www.lustre.org/ [ 7714.178593] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 7714.184710] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7715.345176] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7716.673029] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing set_default_debug -1 all [ 7719.550602] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7719.687552] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7720.998506] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing set_default_debug -1 all [ 7721.959910] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 7724.202585] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7724.323352] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7725.346481] LustreError: 321304:0:(ldlm_lib.c:1180:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7725.349822] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000400:36 to 0x280000400:97) [ 7725.352852] LustreError: 321304:0:(ldlm_lib.c:1180:target_handle_connect()) Skipped 2 previous similar messages [ 7726.091369] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing set_default_debug -1 all [ 7728.846848] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7728.918731] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 7729.961899] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:65) [ 7730.698945] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing set_default_debug -1 all [ 7734.825233] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7740.453932] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 7745.504245] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7745.506713] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7745.507835] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7745.511912] Lustre: Skipped 3 previous similar messages [ 7745.516072] Lustre: Skipped 3 previous similar messages [ 7749.979503] Lustre: server umount lustre-MDT0000 complete [ 7750.625636] LustreError: 319848:0:(ldlm_lib.c:1180:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7750.630991] LustreError: 319848:0:(ldlm_lib.c:1180:target_handle_connect()) Skipped 3 previous similar messages [ 7751.151758] LustreError: 322789:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1777501624 with bad export cookie 8391921797995394974 [ 7751.154934] LustreError: MGC192.168.203.158@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7751.155343] LustreError: 322789:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 7751.276176] Lustre: server umount lustre-MDT0001 complete [ 7762.945923] Lustre: server umount lustre-OST0000 complete [ 7774.430214] Lustre: server umount lustre-OST0001 complete [ 7779.496092] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing load_modules_local [ 7782.754358] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7782.881530] LustreError: 325024:0:(ldlm_lib.c:1180:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7782.899349] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7784.117747] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing set_default_debug -1 all [ 7786.816562] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7788.253468] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing set_default_debug -1 all [ 7789.253386] Lustre: 326130:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 7791.299261] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7791.403364] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7791.405126] Lustre: Skipped 1 previous similar message [ 7792.419063] LustreError: 326484:0:(ldlm_lib.c:1180:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7792.422124] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000400:36 to 0x280000400:129) [ 7793.172295] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing set_default_debug -1 all [ 7795.833425] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7796.970729] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:97) [ 7797.644555] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing set_default_debug -1 all [ 7800.523250] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7801.649525] Lustre: 327965:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 7807.457082] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7807.460071] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7807.462270] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7807.466918] Lustre: Skipped 1 previous similar message [ 7807.471278] Lustre: Skipped 3 previous similar messages [ 7808.725105] Lustre: server umount lustre-MDT0000 complete [ 7809.883200] LustreError: 325005:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1777501682 with bad export cookie 8391921797995396633 [ 7809.885770] LustreError: MGC192.168.203.158@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7809.886341] LustreError: 325005:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 7809.990245] Lustre: server umount lustre-MDT0001 complete [ 7821.819687] Lustre: server umount lustre-OST0000 complete [ 7832.153522] Lustre: server umount lustre-OST0001 complete [ 7833.234680] Lustre: DEBUG MARKER: SKIP: conf-sanity test_114 skipping SLOW test 114 [ 7833.715094] Lustre: DEBUG MARKER: SKIP: conf-sanity test_115 skipping excluded test 115 [ 7834.249485] Lustre: DEBUG MARKER: == conf-sanity test 116: big size MDT support ============ 18:28:26 (1777501706) [ 7840.930185] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing load_modules_local [ 7843.910297] loop0: detected capacity change from 0 to 1099511627776 [ 7844.067659] SGI XFS with ACLs, security attributes, quota, no debug enabled [ 7844.071111] *** VALIDATE xfs *** [ 7844.077555] XFS (loop0): Mounting V5 Filesystem [ 7844.086427] XFS (loop0): Ending clean mount [ 7844.525086] loop1: detected capacity change from 0 to 18691697672192 [ 7863.921728] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing set_hostid [ 7866.938266] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing load_modules_local [ 7878.570848] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7878.685985] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 7878.699606] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 7878.741978] Lustre: lustre-MDT0000: new disk, initializing [ 7878.779723] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7878.783600] Lustre: Skipped 1 previous similar message [ 7878.791527] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 7880.144210] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing set_default_debug -1 all [ 7884.094559] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7884.132626] Lustre: 333025:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 7884.153211] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 7884.154894] Lustre: Skipped 1 previous similar message [ 7884.186535] Lustre: lustre-MDT0001: new disk, initializing [ 7884.221388] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 7884.224106] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 7885.413276] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing set_default_debug -1 all [ 7888.387353] Lustre: DEBUG MARKER: oleg358-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 7889.601890] Lustre: DEBUG MARKER: oleg358-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 7891.789058] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7891.878125] Lustre: lustre-OST0000: new disk, initializing [ 7891.879973] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 7893.623180] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 7893.626592] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 7893.658427] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 7893.752962] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing set_default_debug -1 all [ 7896.714132] Lustre: DEBUG MARKER: oleg358-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 7898.052886] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 7898.133658] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 7899.328817] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 7899.392997] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 7904.225497] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 7904.226743] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7904.228540] LustreError: Skipped 1 previous similar message [ 7904.229325] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7904.233277] Lustre: Skipped 3 previous similar messages [ 7906.064479] Lustre: server umount lustre-OST0000 complete [ 7909.344454] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7909.345193] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7909.348740] Lustre: Skipped 1 previous similar message [ 7909.351562] Lustre: Skipped 2 previous similar messages [ 7913.366737] Lustre: server umount lustre-MDT0000 complete [ 7914.463819] LustreError: 333035:0:(ldlm_lib.c:1180:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7914.468408] LustreError: 333035:0:(ldlm_lib.c:1180:target_handle_connect()) Skipped 1 previous similar message [ 7914.546051] LustreError: 333017:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1777501787 with bad export cookie 8391921797995398173 [ 7914.549100] LustreError: MGC192.168.203.158@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7914.550846] LustreError: 333017:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 5 previous similar messages [ 7914.664904] Lustre: server umount lustre-MDT0001 complete [ 7917.176772] XFS (loop0): Unmounting Filesystem [ 7918.649635] Lustre: DEBUG MARKER: == conf-sanity test 117: lctl get_param return errors properly ========================================================== 18:29:50 (1777501790) [ 7920.750440] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7920.927104] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7920.929760] Lustre: Skipped 2 previous similar messages [ 7922.118282] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing set_default_debug -1 all [ 7924.810306] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7926.130648] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing set_default_debug -1 all [ 7927.853832] Lustre: DEBUG MARKER: oleg358-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 7928.968479] Lustre: DEBUG MARKER: oleg358-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 7931.086706] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7932.821977] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing set_default_debug -1 all [ 7934.507566] Lustre: DEBUG MARKER: oleg358-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 7937.503858] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 7937.507541] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7937.512222] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7943.569023] Lustre: server umount lustre-OST0000 complete [ 7945.695862] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7945.699033] LustreError: Skipped 1 previous similar message [ 7945.701015] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7945.705817] Lustre: Skipped 1 previous similar message [ 7945.708470] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7945.710911] Lustre: Skipped 2 previous similar messages [ 7950.932880] Lustre: server umount lustre-MDT0000 complete [ 7952.086443] LustreError: 337198:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1777501825 with bad export cookie 8391921797995399055 [ 7952.089678] LustreError: MGC192.168.203.158@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7952.091937] LustreError: 337198:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 7952.196459] Lustre: server umount lustre-MDT0001 complete [ 7955.567680] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing unload_modules_local [ 7956.537765] Key type lgssc unregistered [ 7956.668305] LNet: 340441:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 7956.671218] LNetError: 340441:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 7956.678276] LNet: Removed LNI 192.168.203.158@tcp [ 7956.967121] Key type .llcrypt unregistered [ 7956.968808] Key type ._llcrypt unregistered [ 7966.488614] Key type ._llcrypt registered [ 7966.489550] Key type .llcrypt registered [ 7966.519183] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing load_modules_local [ 7966.954261] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 7966.963726] alg: No test for adler32 (adler32-zlib) [ 7967.820060] Lustre: Lustre: Build Version: 2.17.52_93_g6cdb58a [ 7967.902639] LNet: Added LNI 192.168.203.158@tcp [8/256/0/180] [ 7969.487165] Key type lgssc registered [ 7969.845151] Lustre: Echo OBD driver; http://www.lustre.org/ [ 7972.455884] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 7972.463486] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7973.613905] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7974.830900] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing set_default_debug -1 all [ 7977.626612] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7977.745217] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7979.011080] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing set_default_debug -1 all [ 7980.785726] Lustre: DEBUG MARKER: oleg358-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 7981.955921] Lustre: DEBUG MARKER: oleg358-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 7984.187033] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7984.293114] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7986.063901] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing set_default_debug -1 all [ 7987.831350] Lustre: DEBUG MARKER: oleg358-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 8008.671709] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [ 8008.673682] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8008.677532] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 8009.618980] Lustre: lustre-MDT0001: Not available for connect from 192.168.203.58@tcp (stopping) [ 8010.209974] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8013.005795] Lustre: server umount lustre-MDT0001 complete [ 8014.732332] LustreError: 341963:0:(ldlm_lib.c:1180:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.203.58@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8014.739693] LustreError: 341963:0:(ldlm_lib.c:1180:target_handle_connect()) Skipped 2 previous similar messages [ 8015.328920] LustreError: 341968:0:(ldlm_lib.c:1180:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8015.333987] LustreError: 341968:0:(ldlm_lib.c:1180:target_handle_connect()) Skipped 1 previous similar message [ 8016.264172] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8016.296164] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [ 8016.298310] Lustre: Found index 1 for lustre-MDT0001, updating log [ 8016.310024] Lustre: 341959:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 8016.413258] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8017.667920] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing set_default_debug -1 all [ 8021.476623] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 8022.499964] LustreError: 345344:0:(obd_mount.c:225:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [ 8022.503149] LustreError: 345344:0:(tgt_mount.c:668:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [ 8022.507625] LustreError: 341961:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [ 8022.522829] LustreError: 345347:0:(obd_config.c:2103:class_config_llog_handler()) MGC192.168.203.158@tcp: cfg command failed: rc = -17 [ 8022.525849] Lustre: cmd=cf001 0:lustre-MDT0001-osp-MDT0000 1:osp 2:lustre-MDT0000-mdtlov_UUID [ 8022.525849] [ 8321.938865] Lustre: lustre-MDT0001: Not available for connect from 192.168.203.58@tcp (stopping) [ 8321.941448] Lustre: Skipped 2 previous similar messages [ 8323.552764] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [ 8323.554283] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8323.559974] Lustre: Skipped 1 previous similar message [ 8326.740994] Lustre: server umount lustre-MDT0001 complete [ 8327.052817] LustreError: 350521:0:(ldlm_lib.c:1180:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.203.58@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8330.365576] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8330.399848] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [ 8330.401705] Lustre: Found index 1 for lustre-MDT0001, updating log [ 8330.414374] Lustre: 341956:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 8330.520987] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8331.866409] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing set_default_debug -1 all [ 8335.844573] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 8337.894939] LustreError: 358100:0:(obd_mount.c:225:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [ 8337.898333] LustreError: 358100:0:(tgt_mount.c:668:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [ 8337.901871] LustreError: 341961:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [ 8337.904893] LustreError: 341961:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [ 8337.922604] LustreError: 358103:0:(obd_config.c:761:class_setup()) osp: device 8 already setup: rc = -17 [ 8337.925727] LustreError: 358103:0:(obd_config.c:866:class_detach()) OBD device 8 still set up [ 8337.929049] LustreError: 358103:0:(obd_config.c:2103:class_config_llog_handler()) MGC192.168.203.158@tcp: cfg command failed: rc = -17 [ 8337.932478] Lustre: cmd=cf003 0:lustre-MDT0001-osp-MDT0000 1:lustre-MDT0001_UUID 2:192.168.203.158@tcp [ 8337.932478] [ 8634.261628] Lustre: lustre-MDT0001: Not available for connect from 192.168.203.58@tcp (stopping) [ 8634.263804] Lustre: Skipped 2 previous similar messages [ 8637.921636] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [ 8637.922105] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8637.927576] Lustre: Skipped 1 previous similar message [ 8639.372237] Lustre: lustre-MDT0001: Not available for connect from 192.168.203.58@tcp (stopping) [ 8639.374452] Lustre: Skipped 2 previous similar messages [ 8640.080915] Lustre: server umount lustre-MDT0001 complete [ 8643.041296] LustreError: 341968:0:(ldlm_lib.c:1180:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8643.046850] LustreError: 341968:0:(ldlm_lib.c:1180:target_handle_connect()) Skipped 3 previous similar messages [ 8643.542894] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8643.576449] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [ 8643.578517] Lustre: Found index 1 for lustre-MDT0001, updating log [ 8643.589751] Lustre: 341958:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 8643.702939] Lustre: lustre-MDT0001: Imperative Recovery enabled, recovery window shrunk from 60-180 down to 60-180 [ 8645.069053] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing set_default_debug -1 all [ 8649.192231] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 8650.215103] LustreError: 370714:0:(obd_mount.c:225:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [ 8650.218036] LustreError: 370714:0:(tgt_mount.c:668:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [ 8650.222597] LustreError: 341961:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [ 8650.225433] LustreError: 341961:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [ 8650.251944] LustreError: 370721:0:(lod_lov.c:212:lod_add_device()) lustre-MDT0001-osp-MDT0000: cannot connect to next dev lustre-MDT0001_UUID (-114) [ 8650.257232] LustreError: 370721:0:(obd_config.c:2103:class_config_llog_handler()) MGC192.168.203.158@tcp: cfg command failed: rc = -114 [ 8650.260340] Lustre: cmd=cf014 0:lustre-MDT0000-mdtlov 1:lustre-MDT0001_UUID 2:1 3:1 [ 8650.260340] [ 8960.992615] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8960.997043] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8961.002574] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8965.528455] Lustre: server umount lustre-MDT0000 complete [ 8966.624123] LustreError: 341964:0:(ldlm_lib.c:1180:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8966.629355] LustreError: 341964:0:(ldlm_lib.c:1180:target_handle_connect()) Skipped 2 previous similar messages [ 8966.689294] LustreError: 341950:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1777502839 with bad export cookie 13586099099708001487 [ 8966.691239] LustreError: MGC192.168.203.158@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8966.693667] LustreError: 341950:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 8971.744416] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8971.749222] Lustre: Skipped 2 previous similar messages [ 8972.865222] Lustre: server umount lustre-MDT0001 complete [ 8974.302695] Lustre: server umount lustre-OST0000 complete [ 8977.152568] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing set_hostid [ 8980.261225] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing load_modules_local [ 8991.895352] Lustre: DEBUG MARKER: == conf-sanity test 120: cross-target rename should not create bad symlinks ========================================================== 18:47:43 (1777502863) [ 8994.095420] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8994.194730] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 8994.207943] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 8994.241239] Lustre: lustre-MDT0000: new disk, initializing [ 8994.268791] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8994.274997] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 8995.513071] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing set_default_debug -1 all [ 8999.426975] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8999.468957] Lustre: 385385:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 8999.492724] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 8999.495235] Lustre: Skipped 1 previous similar message [ 8999.527111] Lustre: lustre-MDT0001: new disk, initializing [ 8999.555651] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8999.567583] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 8999.571451] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 9000.721097] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing set_default_debug -1 all [ 9003.623655] Lustre: DEBUG MARKER: oleg358-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 9004.712736] Lustre: DEBUG MARKER: oleg358-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 9006.799549] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9006.889048] Lustre: lustre-OST0000: new disk, initializing [ 9006.890610] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 9006.917810] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9007.861330] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 9007.865244] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 9007.897338] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 9008.693076] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing set_default_debug -1 all [ 9011.687359] Lustre: DEBUG MARKER: oleg358-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 9015.263763] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 9015.266865] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9015.272116] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9015.274479] Lustre: Skipped 4 previous similar messages [ 9020.255807] Lustre: server umount lustre-MDT0000 complete [ 9021.477090] LustreError: 385378:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1777502894 with bad export cookie 13586099099708003426 [ 9021.480222] LustreError: MGC192.168.203.158@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9021.481506] LustreError: 385378:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 9021.595721] Lustre: server umount lustre-MDT0001 complete [ 9033.121943] Lustre: server umount lustre-OST0000 complete [ 9041.969570] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing set_hostid [ 9044.955684] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing load_modules_local [ 9054.466930] Lustre: DEBUG MARKER: == conf-sanity test complete, duration 8751 sec ========== 18:48:46 (1777502926) [ 9054.974600] Lustre: DEBUG MARKER: === conf-sanity: start cleanup 18:48:46 (1777502926) === [ 9056.036880] Lustre: DEBUG MARKER: === conf-sanity: finish cleanup 18:48:47 (1777502927) === [ 9069.385421] Lustre: DEBUG MARKER: oleg358-server.virtnet: executing unload_modules_local [ 9070.495237] Key type lgssc unregistered [ 9070.631334] LNet: 393317:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 9070.633824] LNetError: 393317:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 9070.643298] LNet: Removed LNI 192.168.203.158@tcp [ 9070.942701] Key type .llcrypt unregistered [ 9070.944403] Key type ._llcrypt unregistered