[ 0.000000] Linux version 4.18.0rh8.10-debug (green@maintenance) (gcc version 8.5.0 20210514 (Red Hat 8.5.0-26) (GCC)) #2 SMP Mon Jul 14 01:24:22 EDT 2025 [ 0.000000] Command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers' [ 0.000000] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256 [ 0.000000] x86/fpu: Enabled xstate features 0x7, context size is 832 bytes, using 'standard' format. [ 0.000000] signal: max sigframe size: 1776 [ 0.000000] BIOS-provided physical RAM map: [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x00000000bffcdfff] usable [ 0.000000] BIOS-e820: [mem 0x00000000bffce000-0x00000000bfffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x0000000146dfffff] usable [ 0.000000] NX (Execute Disable) protection: active [ 0.000000] SMBIOS 2.8 present. [ 0.000000] DMI: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-8.fc42 06/10/2025 [ 0.000000] Hypervisor detected: KVM [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00 [ 0.000000] kvm-clock: using sched offset of 505876479 cycles [ 0.000000] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns [ 0.000000] tsc: Detected 2399.998 MHz processor [ 0.000000] last_pfn = 0x146e00 max_arch_pfn = 0x400000000 [ 0.000000] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT [ 0.000000] last_pfn = 0xbffce max_arch_pfn = 0x400000000 [ 0.000000] found SMP MP-table at [mem 0x000f54b0-0x000f54bf] [ 0.000000] RAMDISK: [mem 0xbcc54000-0xbffbffff] [ 0.000000] ACPI: Early table checksum verification disabled [ 0.000000] ACPI: RSDP 0x00000000000F52D0 000014 (v00 BOCHS ) [ 0.000000] ACPI: RSDT 0x00000000BFFE2439 000034 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACP 0x00000000BFFE22D5 000074 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: DSDT 0x00000000BFFE0040 002295 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACS 0x00000000BFFE0000 000040 [ 0.000000] ACPI: APIC 0x00000000BFFE2349 000090 (v03 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: HPET 0x00000000BFFE23D9 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: WAET 0x00000000BFFE2411 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: Reserving FACP table memory at [mem 0xbffe22d5-0xbffe2348] [ 0.000000] ACPI: Reserving DSDT table memory at [mem 0xbffe0040-0xbffe22d4] [ 0.000000] ACPI: Reserving FACS table memory at [mem 0xbffe0000-0xbffe003f] [ 0.000000] ACPI: Reserving APIC table memory at [mem 0xbffe2349-0xbffe23d8] [ 0.000000] ACPI: Reserving HPET table memory at [mem 0xbffe23d9-0xbffe2410] [ 0.000000] ACPI: Reserving WAET table memory at [mem 0xbffe2411-0xbffe2438] [ 0.000000] No NUMA configuration found [ 0.000000] Faking a node at [mem 0x0000000000000000-0x0000000146dfffff] [ 0.000000] NODE_DATA(0) allocated [mem 0x1465a3000-0x1465cdfff] [ 0.000000] Reserving 256MB of memory at 2752MB for crashkernel (System RAM: 4205MB) [ 0.000000] Zone ranges: [ 0.000000] DMA [mem 0x0000000000001000-0x0000000000ffffff] [ 0.000000] DMA32 [mem 0x0000000001000000-0x00000000ffffffff] [ 0.000000] Normal [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Device empty [ 0.000000] Movable zone start for each node [ 0.000000] Early memory node ranges [ 0.000000] node 0: [mem 0x0000000000001000-0x000000000009efff] [ 0.000000] node 0: [mem 0x0000000000100000-0x00000000bffcdfff] [ 0.000000] node 0: [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Zeroed struct page in unavailable ranges: 4756 pages [ 0.000000] Initmem setup node 0 [mem 0x0000000000001000-0x0000000146dfffff] [ 0.000000] ACPI: PM-Timer IO Port: 0x608 [ 0.000000] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1]) [ 0.000000] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23 [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level) [ 0.000000] Using ACPI (MADT) for SMP configuration information [ 0.000000] ACPI: HPET id: 0x8086a201 base: 0xfed00000 [ 0.000000] TSC deadline timer available [ 0.000000] smpboot: Allowing 4 CPUs, 0 hotplug CPUs [ 0.000000] kvm-guest: KVM setup pv remote TLB flush [ 0.000000] kvm-guest: setup PV sched yield [ 0.000000] PM: Registered nosave memory: [mem 0x00000000-0x00000fff] [ 0.000000] PM: Registered nosave memory: [mem 0x0009f000-0x0009ffff] [ 0.000000] PM: Registered nosave memory: [mem 0x000a0000-0x000effff] [ 0.000000] PM: Registered nosave memory: [mem 0x000f0000-0x000fffff] [ 0.000000] PM: Registered nosave memory: [mem 0xbffce000-0xbfffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xc0000000-0xfeffbfff] [ 0.000000] PM: Registered nosave memory: [mem 0xfeffc000-0xfeffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xff000000-0xfffbffff] [ 0.000000] PM: Registered nosave memory: [mem 0xfffc0000-0xffffffff] [ 0.000000] [mem 0xc0000000-0xfeffbfff] available for PCI devices [ 0.000000] Booting paravirtualized kernel on KVM [ 0.000000] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns [ 0.000000] setup_percpu: NR_CPUS:8192 nr_cpumask_bits:4 nr_cpu_ids:4 nr_node_ids:1 [ 0.000000] percpu: Embedded 63 pages/cpu s221184 r8192 d28672 u524288 [ 0.000000] kvm-guest: PV spinlocks enabled [ 0.000000] PV qspinlock hash table entries: 256 (order: 0, 4096 bytes, linear) [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 1059606 [ 0.000000] Policy zone: Normal [ 0.000000] Kernel command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] Specific versions of hardware are certified with Red Hat Enterprise Linux 8. Please see the list of hardware certified with Red Hat Enterprise Linux 8 at https://catalog.redhat.com. [ 0.000000] audit: disabled (until reboot) [ 0.000000] software IO TLB: area num 4. [ 0.000000] Memory: 2895240K/4306352K available (18435K kernel code, 11221K rwdata, 7248K rodata, 2908K init, 18040K bss, 524584K reserved, 0K cma-reserved) [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1 [ 0.000000] kmemleak: Kernel memory leak detector disabled [ 0.000000] ftrace: allocating 41240 entries in 162 pages [ 0.000000] ftrace: allocated 162 pages with 3 groups [ 0.000000] rcu: Hierarchical RCU implementation. [ 0.000000] rcu: RCU event tracing is enabled. [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=8192 to nr_cpu_ids=4. [ 0.000000] rcu: RCU callback double-/use-after-free debug enabled. [ 0.000000] Rude variant of Tasks RCU enabled. [ 0.000000] Tracing variant of Tasks RCU enabled. [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies. [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4 [ 0.000000] NR_IRQS: 524544, nr_irqs: 456, preallocated irqs: 16 [ 0.000000] random: get_random_bytes called from start_kernel+0x622/0x9a8 with crng_init=0 [ 0.001000] Console: colour *CGA 80x25 [ 0.001000] printk: console [ttyS1] enabled [ 0.001000] ACPI: Core revision 20220331 [ 0.001000] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns [ 0.001013] APIC: Switch to symmetric I/O mode setup [ 0.003297] x2apic enabled [ 0.004011] Switched APIC routing to physical x2apic. [ 0.005016] kvm-guest: setup PV IPIs [ 0.008749] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1 [ 0.009000] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 0.009022] Calibrating delay loop (skipped) preset value.. 4799.99 BogoMIPS (lpj=2399998) [ 0.010016] pid_max: default: 32768 minimum: 301 [ 0.012124] LSM: Security Framework initializing [ 0.013056] Yama: becoming mindful. [ 0.014043] SELinux: Initializing. [ 0.016048] *** VALIDATE selinux *** [ 0.024723] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, vmalloc) [ 0.029393] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, vmalloc) [ 0.030181] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.031138] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.032135] *** VALIDATE tmpfs *** [ 0.034522] *** VALIDATE proc *** [ 0.036116] *** VALIDATE cgroup *** [ 0.037012] *** VALIDATE cgroup2 *** [ 0.038285] x86/cpu: User Mode Instruction Prevention (UMIP) activated [ 0.039200] Last level iTLB entries: 4KB 0, 2MB 0, 4MB 0 [ 0.040010] Last level dTLB entries: 4KB 0, 2MB 0, 4MB 0, 1GB 0 [ 0.042035] Spectre V2 : User space: Vulnerable [ 0.043011] Speculative Store Bypass: Vulnerable [ 0.045783] debug: unmapping init [mem 0xffffffffa4a59000-0xffffffffa4a60fff] [ 0.047887] smpboot: CPU0: Intel(R) Xeon(R) CPU E5-2695 v2 @ 2.40GHz (family: 0x6, model: 0x3e, stepping: 0x4) [ 0.048817] Performance Events: IvyBridge events, full-width counters, Intel PMU driver. [ 0.049030] ... version: 2 [ 0.050015] ... bit width: 48 [ 0.051030] ... generic registers: 4 [ 0.052017] ... value mask: 0000ffffffffffff [ 0.053016] ... max period: 00007fffffffffff [ 0.054016] ... fixed-purpose events: 3 [ 0.055019] ... event mask: 000000070000000f [ 0.056361] rcu: Hierarchical SRCU implementation. [ 0.058615] smp: Bringing up secondary CPUs ... [ 0.059701] x86: Booting SMP configuration: [ 0.060031] .... node #0, CPUs: #1 #2 #3 [ 0.065222] smp: Brought up 1 node, 4 CPUs [ 0.067021] smpboot: Max logical packages: 1 [ 0.068021] smpboot: Total of 4 processors activated (19199.98 BogoMIPS) [ 0.137800] node 0 deferred pages initialised in 67ms [ 0.142064] devtmpfs: initialized [ 0.143279] x86/mm: Memory block size: 128MB [ 0.147000] gcov: version magic: 0x41383552 [ 0.149300] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns [ 0.153125] futex hash table entries: 1024 (order: 4, 65536 bytes, vmalloc) [ 0.155338] pinctrl core: initialized pinctrl subsystem [ 0.157211] [ 0.157761] ************************************************************* [ 0.160015] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.163014] ** ** [ 0.166014] ** IOMMU DebugFS SUPPORT HAS BEEN ENABLED IN THIS KERNEL ** [ 0.168013] ** ** [ 0.171017] ** This means that this kernel is built to expose internal ** [ 0.173014] ** IOMMU data structures, which may compromise security on ** [ 0.175015] ** your system. ** [ 0.177016] ** ** [ 0.179016] ** If you see this message and you are not debugging the ** [ 0.181013] ** kernel, report this immediately to your vendor! ** [ 0.183013] ** ** [ 0.186014] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.188012] ************************************************************* [ 0.190743] NET: Registered protocol family 16 [ 0.192641] DMA: preallocated 512 KiB GFP_KERNEL pool for atomic allocations [ 0.195066] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations [ 0.198101] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations [ 0.202025] cpuidle: using governor menu [ 0.203849] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5 [ 0.206437] PCI: Using configuration type 1 for base access [ 0.208136] core: PMU erratum BJ122, BV98, HSD29 worked around, HT is on [ 0.216144] HugeTLB registered 1.00 GiB page size, pre-allocated 0 pages [ 0.217037] HugeTLB registered 2.00 MiB page size, pre-allocated 0 pages [ 0.219041] cryptd: max_cpu_qlen set to 1000 [ 0.220193] ACPI: Added _OSI(Module Device) [ 0.222025] ACPI: Added _OSI(Processor Device) [ 0.223026] ACPI: Added _OSI(3.0 _SCP Extensions) [ 0.225017] ACPI: Added _OSI(Processor Aggregator Device) [ 0.229230] ACPI: 1 ACPI AML tables successfully acquired and loaded [ 0.234430] ACPI: Interpreter enabled [ 0.235059] ACPI: PM: (supports S0 S3 S4 S5) [ 0.237011] ACPI: Using IOAPIC for interrupt routing [ 0.238081] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug [ 0.240447] ACPI: Enabled 2 GPEs in block 00 to 0F [ 0.249000] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff]) [ 0.251049] acpi PNP0A03:00: _OSC: OS supports [ASPM ClockPM Segments MSI HPX-Type3] [ 0.253021] acpi PNP0A03:00: _OSC: not requesting OS control; OS requires [ExtendedConfig ASPM ClockPM MSI] [ 0.256112] acpi PNP0A03:00: fail to add MMCONFIG information, can't access extended PCI configuration space under this bridge. [ 0.260299] acpiphp: Slot [2] registered [ 0.261113] acpiphp: Slot [5] registered [ 0.262330] acpiphp: Slot [6] registered [ 0.263139] acpiphp: Slot [7] registered [ 0.264061] acpiphp: Slot [8] registered [ 0.266082] acpiphp: Slot [9] registered [ 0.267102] acpiphp: Slot [10] registered [ 0.268044] acpiphp: Slot [3] registered [ 0.269000] acpiphp: Slot [4] registered [ 0.269000] acpiphp: Slot [11] registered [ 0.271134] acpiphp: Slot [12] registered [ 0.272071] acpiphp: Slot [13] registered [ 0.273040] acpiphp: Slot [14] registered [ 0.274076] acpiphp: Slot [15] registered [ 0.275084] acpiphp: Slot [16] registered [ 0.276075] acpiphp: Slot [17] registered [ 0.277089] acpiphp: Slot [18] registered [ 0.278083] acpiphp: Slot [19] registered [ 0.279075] acpiphp: Slot [20] registered [ 0.280057] acpiphp: Slot [21] registered [ 0.282086] acpiphp: Slot [22] registered [ 0.283065] acpiphp: Slot [23] registered [ 0.284107] acpiphp: Slot [24] registered [ 0.285085] acpiphp: Slot [25] registered [ 0.286086] acpiphp: Slot [26] registered [ 0.287056] acpiphp: Slot [27] registered [ 0.288080] acpiphp: Slot [28] registered [ 0.289061] acpiphp: Slot [29] registered [ 0.289977] acpiphp: Slot [30] registered [ 0.291092] acpiphp: Slot [31] registered [ 0.292092] PCI host bridge to bus 0000:00 [ 0.292963] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window] [ 0.294020] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window] [ 0.296017] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window] [ 0.298023] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window] [ 0.300024] pci_bus 0000:00: root bus resource [mem 0xe0000000000-0xe007fffffff window] [ 0.302023] pci_bus 0000:00: root bus resource [bus 00-ff] [ 0.303173] pci 0000:00:00.0: [8086:1237] type 00 class 0x060000 [ 0.306510] pci 0000:00:01.0: [8086:7000] type 00 class 0x060100 [ 0.310229] pci 0000:00:01.1: [8086:7010] type 00 class 0x010180 [ 0.318015] pci 0000:00:01.1: reg 0x20: [io 0xc320-0xc32f] [ 0.322743] pci 0000:00:01.1: legacy IDE quirk: reg 0x10: [io 0x01f0-0x01f7] [ 0.325019] pci 0000:00:01.1: legacy IDE quirk: reg 0x14: [io 0x03f6] [ 0.327012] pci 0000:00:01.1: legacy IDE quirk: reg 0x18: [io 0x0170-0x0177] [ 0.328013] pci 0000:00:01.1: legacy IDE quirk: reg 0x1c: [io 0x0376] [ 0.331217] pci 0000:00:01.3: [8086:7113] type 00 class 0x068000 [ 0.333683] pci 0000:00:01.3: quirk: [io 0x0600-0x063f] claimed by PIIX4 ACPI [ 0.335033] pci 0000:00:01.3: quirk: [io 0x0700-0x070f] claimed by PIIX4 SMB [ 0.337634] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 [ 0.341017] pci 0000:00:02.0: reg 0x10: [io 0xc300-0xc31f] [ 0.351015] pci 0000:00:02.0: reg 0x20: [mem 0xe0000000000-0xe0000003fff 64bit pref] [ 0.356014] pci 0000:00:02.0: reg 0x30: [mem 0xfeb80000-0xfebbffff pref] [ 0.360235] pci 0000:00:05.0: [1af4:1001] type 00 class 0x010000 [ 0.367022] pci 0000:00:05.0: reg 0x10: [io 0xc000-0xc07f] [ 0.375022] pci 0000:00:05.0: reg 0x14: [mem 0xfebc0000-0xfebc0fff] [ 0.388030] pci 0000:00:05.0: reg 0x20: [mem 0xe0000004000-0xe0000007fff 64bit pref] [ 0.398006] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 [ 0.403018] pci 0000:00:06.0: reg 0x10: [io 0xc080-0xc0ff] [ 0.408015] pci 0000:00:06.0: reg 0x14: [mem 0xfebc1000-0xfebc1fff] [ 0.420017] pci 0000:00:06.0: reg 0x20: [mem 0xe0000008000-0xe000000bfff 64bit pref] [ 0.427013] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 [ 0.431014] pci 0000:00:07.0: reg 0x10: [io 0xc100-0xc17f] [ 0.435019] pci 0000:00:07.0: reg 0x14: [mem 0xfebc2000-0xfebc2fff] [ 0.445016] pci 0000:00:07.0: reg 0x20: [mem 0xe000000c000-0xe000000ffff 64bit pref] [ 0.451280] pci 0000:00:08.0: [1af4:1001] type 00 class 0x010000 [ 0.456012] pci 0000:00:08.0: reg 0x10: [io 0xc180-0xc1ff] [ 0.459016] pci 0000:00:08.0: reg 0x14: [mem 0xfebc3000-0xfebc3fff] [ 0.471015] pci 0000:00:08.0: reg 0x20: [mem 0xe0000010000-0xe0000013fff 64bit pref] [ 0.482821] pci 0000:00:09.0: [1af4:1001] type 00 class 0x010000 [ 0.493019] pci 0000:00:09.0: reg 0x10: [io 0xc200-0xc27f] [ 0.500019] pci 0000:00:09.0: reg 0x14: [mem 0xfebc4000-0xfebc4fff] [ 0.515020] pci 0000:00:09.0: reg 0x20: [mem 0xe0000014000-0xe0000017fff 64bit pref] [ 0.525000] pci 0000:00:0a.0: [1af4:1001] type 00 class 0x010000 [ 0.531015] pci 0000:00:0a.0: reg 0x10: [io 0xc280-0xc2ff] [ 0.536015] pci 0000:00:0a.0: reg 0x14: [mem 0xfebc5000-0xfebc5fff] [ 0.547018] pci 0000:00:0a.0: reg 0x20: [mem 0xe0000018000-0xe000001bfff 64bit pref] [ 0.556928] ACPI: PCI: Interrupt link LNKA configured for IRQ 10 [ 0.560385] ACPI: PCI: Interrupt link LNKB configured for IRQ 10 [ 0.562326] ACPI: PCI: Interrupt link LNKC configured for IRQ 11 [ 0.564339] ACPI: PCI: Interrupt link LNKD configured for IRQ 11 [ 0.566309] ACPI: PCI: Interrupt link LNKS configured for IRQ 9 [ 0.572017] iommu: Default domain type: Passthrough [ 0.573467] SCSI subsystem initialized [ 0.574269] ACPI: bus type USB registered [ 0.575000] usbcore: registered new interface driver usbfs [ 0.577095] usbcore: registered new interface driver hub [ 0.578091] usbcore: registered new device driver usb [ 0.580138] pps_core: LinuxPPS API ver. 1 registered [ 0.581009] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti [ 0.583083] PTP clock support registered [ 0.585145] EDAC MC: Ver: 3.0.0 [ 0.587105] PCI: Using ACPI for IRQ routing [ 0.588561] NetLabel: Initializing [ 0.590013] NetLabel: domain hash size = 128 [ 0.592013] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO [ 0.593068] NetLabel: unlabeled traffic allowed by default [ 0.595136] vgaarb: loaded [ 0.596259] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0 [ 0.598017] hpet0: 3 comparators, 64-bit 100.000000 MHz counter [ 0.606000] clocksource: Switched to clocksource kvm-clock [ 0.718703] VFS: Disk quotas dquot_6.6.0 [ 0.720372] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) [ 0.723120] *** VALIDATE ramfs *** [ 0.724434] *** VALIDATE hugetlbfs *** [ 0.726859] pnp: PnP ACPI init [ 0.729401] pnp: PnP ACPI: found 6 devices [ 0.745424] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns [ 0.748918] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window] [ 0.751041] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window] [ 0.753044] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window] [ 0.755012] pci_bus 0000:00: resource 7 [mem 0xc0000000-0xfebfffff window] [ 0.757238] pci_bus 0000:00: resource 8 [mem 0xe0000000000-0xe007fffffff window] [ 0.760090] NET: Registered protocol family 2 [ 0.762659] IP idents hash table entries: 131072 (order: 8, 1048576 bytes, vmalloc) [ 0.767521] tcp_listen_portaddr_hash hash table entries: 4096 (order: 5, 163840 bytes, vmalloc) [ 0.772426] TCP established hash table entries: 65536 (order: 7, 524288 bytes, vmalloc) [ 0.778034] TCP bind hash table entries: 65536 (order: 9, 2097152 bytes, vmalloc) [ 0.781953] TCP: Hash tables configured (established 65536 bind 65536) [ 0.784961] MPTCP token hash table entries: 8192 (order: 6, 393216 bytes, vmalloc) [ 0.788396] UDP hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 0.791370] UDP-Lite hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 0.794625] NET: Registered protocol family 1 [ 0.797462] RPC: Registered named UNIX socket transport module. [ 0.800363] RPC: Registered udp transport module. [ 0.802307] RPC: Registered tcp transport module. [ 0.804187] RPC: Registered tcp NFSv4.1 backchannel transport module. [ 0.807031] NET: Registered protocol family 44 [ 0.808853] pci 0000:00:00.0: Limiting direct PCI/PCI transfers [ 0.811036] pci 0000:00:01.0: PIIX3: Enabling Passive Release [ 0.813217] pci 0000:00:01.0: Activating ISA DMA hang workarounds [ 0.815764] PCI: CLS 0 bytes, default 64 [ 0.818154] Unpacking initramfs... [ 2.216996] debug: unmapping init [mem 0xffff9e11fcc54000-0xffff9e11fffbffff] [ 2.222340] PCI-DMA: Using software bounce buffering for IO (SWIOTLB) [ 2.225717] software IO TLB: mapped [mem 0x00000000a8000000-0x00000000ac000000] (64MB) [ 2.230077] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 2.734779] Initialise system trusted keyrings [ 2.736164] Key type blacklist registered [ 2.738437] workingset: timestamp_bits=36 max_order=20 bucket_order=0 [ 2.746881] zbud: loaded [ 2.750422] *** VALIDATE nfs *** [ 2.751700] *** VALIDATE nfs4 *** [ 2.753403] pstore: using deflate compression [ 2.758418] Platform Keyring initialized [ 2.866977] NET: Registered protocol family 38 [ 2.868994] Key type asymmetric registered [ 2.870532] Asymmetric key parser 'x509' registered [ 2.872322] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 247) [ 2.875425] io scheduler mq-deadline registered [ 2.877236] io scheduler kyber registered [ 2.878852] io scheduler bfq registered [ 2.880668] atomic64_test: passed for x86-64 platform with CX8 and with SSE [ 2.883620] shpchp: Standard Hot Plug PCI Controller Driver version: 0.4 [ 2.886443] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0 [ 2.888437] ACPI: Power Button [PWRF] [ 2.894288] ACPI: \_SB_.LNKB: Enabled at IRQ 10 [ 2.901302] ACPI: \_SB_.LNKA: Enabled at IRQ 11 [ 2.927667] ACPI: \_SB_.LNKC: Enabled at IRQ 11 [ 2.938542] ACPI: \_SB_.LNKD: Enabled at IRQ 10 [ 2.983510] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled [ 3.012674] 00:03: ttyS1 at I/O 0x2f8 (irq = 3, base_baud = 115200) is a 16550A [ 3.042586] 00:04: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A [ 3.046642] Non-volatile memory driver v1.3 [ 3.048426] Linux agpgart interface v0.103 [ 3.079616] virtio_blk virtio1: [vda] 134040 512-byte logical blocks (68.6 MB/65.4 MiB) [ 3.082399] vda: detected capacity change from 0 to 68628480 [ 3.106099] virtio_blk virtio2: [vdb] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) [ 3.109054] vdb: detected capacity change from 0 to 1073741824 [ 3.134897] virtio_blk virtio3: [vdc] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 3.137971] vdc: detected capacity change from 0 to 2621440000 [ 3.159885] virtio_blk virtio4: [vdd] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 3.163165] vdd: detected capacity change from 0 to 2621440000 [ 3.182585] virtio_blk virtio5: [vde] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 3.185628] vde: detected capacity change from 0 to 4294967296 [ 3.202183] virtio_blk virtio6: [vdf] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 3.205429] vdf: detected capacity change from 0 to 4294967296 [ 3.211761] libphy: Fixed MDIO Bus: probed [ 3.223358] usbcore: registered new interface driver usbserial_generic [ 3.225656] usbserial: USB Serial support registered for generic [ 3.229312] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12 [ 3.233905] serio: i8042 KBD port at 0x60,0x64 irq 1 [ 3.236060] serio: i8042 AUX port at 0x60,0x64 irq 12 [ 3.238575] mousedev: PS/2 mouse device common for all mice [ 3.240606] rtc_cmos 00:05: RTC can wake from S4 [ 3.243733] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input1 [ 3.243958] rtc_cmos 00:05: registered as rtc0 [ 3.248451] rtc_cmos 00:05: alarms up to one day, y3k, 242 bytes nvram, hpet irqs [ 3.251213] intel_pstate: CPU model not supported [ 3.251486] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input4 [ 3.257272] hid: raw HID events driver (C) Jiri Kosina [ 3.259378] usbcore: registered new interface driver usbhid [ 3.260125] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input3 [ 3.261124] usbhid: USB HID core driver [ 3.261254] drop_monitor: Initializing network drop monitor service [ 3.267372] Initializing XFRM netlink socket [ 3.269050] NET: Registered protocol family 10 [ 3.271379] Segment Routing with IPv6 [ 3.272713] NET: Registered protocol family 17 [ 3.274400] mpls_gso: MPLS GSO support [ 3.279755] RAS: Correctable Errors collector initialized. [ 3.281838] AVX version of gcm_enc/dec engaged. [ 3.283578] AES CTR mode by8 optimization enabled [ 3.345728] sched_clock: Marking stable (3345710051, 0)->(4235590351, -889880300) [ 3.347705] registered taskstats version 1 [ 3.348692] Loading compiled-in X.509 certificates [ 3.349625] zswap: loaded using pool lzo/zbud [ 3.367774] Key type big_key registered [ 3.379182] Key type encrypted registered [ 3.380857] ima: No TPM chip found, activating TPM-bypass! [ 3.382907] ima: Allocated hash algorithm: sha1 [ 3.384630] ima: No architecture policies found [ 3.386347] evm: Initialising EVM extended attributes: [ 3.388181] evm: security.selinux [ 3.389382] evm: security.ima [ 3.390399] evm: security.capability [ 3.391764] evm: HMAC attrs: 0x1 [ 3.394156] rtc_cmos 00:05: setting system clock to 2026-01-22 09:22:35 UTC (1769073755) [ 3.400621] debug: unmapping init [mem 0xffffffffa5a03000-0xffffffffa5bfffff] [ 3.403378] debug: unmapping init [mem 0xffffffffa4782000-0xffffffffa4a58fff] [ 3.412066] Write protecting the kernel read-only data: 28672k [ 3.415217] debug: unmapping init [mem 0xffffffffa2e03000-0xffffffffa2ffffff] [ 3.417961] debug: unmapping init [mem 0xffffffffa3714000-0xffffffffa37fffff] [ 3.451968] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 3.460276] systemd[1]: Detected virtualization kvm. [ 3.462195] systemd[1]: Detected architecture x86-64. [ 3.463764] systemd[1]: Running in initial RAM disk. Welcome to Rocky Linux 8.10 (Green Obsidian) dracut-049-233.git20240115.el8 (Initramfs)! [ 3.490308] systemd[1]: No hostname configured. [ 3.491736] systemd[1]: Set hostname to . [ 3.493291] random: systemd: uninitialized urandom read (16 bytes read) [ 3.495329] systemd[1]: Initializing machine ID from random generator. [ 3.533949] random: ln: uninitialized urandom read (6 bytes read) [ 3.619262] random: systemd: uninitialized urandom read (16 bytes read) [ 3.621922] systemd[1]: Reached target Swap. [ OK ] Reached target Swap. [ 3.625403] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ 3.629888] systemd[1]: Reached target Slices. [ OK ] Reached target Slices. [ OK ] Reached target Initrd Root Device. [ OK ] Listening on udev Kernel Socket. [ OK ] Listening on udev Control Socket. [ OK ] Reached target Local Encrypted Volumes. [ OK ] Reached target Paths. [ OK ] Reached target Local File Systems. [ OK ] Listening on Journal Socket (/dev/log). [ OK ] Reached target Timers. [ OK ] Listening on Journal Socket. Starting Create list of required st…ce nodes for the current kernel... Starting Setup Virtual Console... [ OK ] Reached target Sockets. Starting Journal Service... Starting Create Volatile Files and Directories... Starting Apply Kernel Variables... [ OK ] Started Memstrack Anylazing Service. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Started Setup Virtual Console. [ OK ] Started Create Volatile Files and Directories. [ OK ] Started Apply Kernel Variables. Starting dracut cmdline hook... Starting Create Static Device Nodes in /dev... [ OK ] Started Journal Service. [ OK ] Started Create Static Device Nodes in /dev. [ OK ] Started dracut cmdline hook. Starting dracut pre-udev hook... [ 4.195495] device-mapper: uevent: version 1.0.3 [ 4.197306] device-mapper: ioctl: 4.46.0-ioctl (2022-02-22) initialised: dm-devel@redhat.com [ OK ] Started dracut pre-udev hook. Starting udev Kernel Device Manager... [ OK ] Started udev Kernel Device Manager. Starting dracut pre-trigger hook... [ OK ] Started dracut pre-trigger hook. Starting udev Coldplug all Devices... Mounting Kernel Configuration File System... [ OK ] Mounted Kernel Configuration File System. [ OK ] Started udev Coldplug all Devices. [ OK ] Reached target System Initialization. [ OK ] Reached target Basic System. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. Starting dracut initqueue hook... [ 4.883902] random: fast init done [ 4.918761] virtio_net virtio0 ens2: renamed from eth0 [ 4.982497] scsi host0: ata_piix [ 5.001876] scsi host1: ata_piix [ 5.003649] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc320 irq 14 [ 5.006355] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc328 irq 15 [ 8.656941] dracut-initqueue[587]: RTNETLINK answers: File exists [ 9.897158] random: crng init done [ 9.898628] random: 7 urandom warning(s) missed due to ratelimiting Starting nbd nbd0... [ OK ] Started nbd nbd0. [ OK ] Started dracut initqueue hook. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Mounting /sysroot... [ 10.303536] EXT4-fs (nbd0): mounted filesystem with ordered data mode. Opts: (null) [ OK ] Mounted /sysroot. [ OK ] Reached target Initrd Root File System. Starting Reload Configuration from the Real Root... [ OK ] Started Reload Configuration from the Real Root. [ OK ] Reached target Initrd File Systems. [ OK ] Reached target Initrd Default Target. Starting dracut pre-pivot and cleanup hook... [ OK ] Started dracut pre-pivot and cleanup hook. Starting Cleaning Up and Shutting Down Daemons... [ OK ] Stopped dracut pre-pivot and cleanup hook. [ OK ] Stopped target Timers. Stopping Hardware RNG Entropy Gatherer Daemon... [ OK ] Stopped target Initrd Default Target. [ OK ] Stopped target Initrd Root Device. [ OK ] Stopped target Remote File Systems. [ OK ] Stopped target Remote File Systems (Pre). [ OK ] Stopped dracut initqueue hook. [ OK ] Stopped Hardware RNG Entropy Gatherer Daemon. [ OK ] Stopped target Basic System. [ OK ] Stopped target Paths. [ OK ] Stopped target Slices. [ OK ] Stopped target System Initialization. [ OK ] Stopped Apply Kernel Variables. [ OK ] Stopped target Swap. [ OK ] Stopped target Local Encrypted Volumes. [ OK ] Stopped Dispatch Password Requests to Console Directory Watch. [ OK ] Stopped udev Coldplug all Devices. [ OK ] Stopped dracut pre-trigger hook. Stopping udev Kernel Device Manager... [ OK ] Stopped Create Volatile Files and Directories. [ OK ] Stopped target Local File Systems. [ OK ] Stopped target Sockets. [ OK ] Started Cleaning Up and Shutting Down Daemons. [ OK ] Stopped udev Kernel Device Manager. [ OK ] Stopped Create Static Device Nodes in /dev. [ OK ] Stopped Create list of required sta…vice nodes for the current kernel. [ OK ] Stopped dracut pre-udev hook. [ OK ] Stopped dracut cmdline hook. [ OK ] Closed udev Kernel Socket. [ OK ] Closed udev Control Socket. Starting Cleanup udevd DB... [ OK ] Started Cleanup udevd DB. [ OK ] Reached target Switch Root. Starting Switch Root... [ 11.473743] printk: systemd: 25 output lines suppressed due to ratelimiting [ 11.741764] SELinux: Disabled at runtime. [ 11.807561] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 11.816685] systemd[1]: Detected virtualization kvm. [ 11.818605] systemd[1]: Detected architecture x86-64. Welcome to Rocky Linux 8.10 (Green Obsidian)! [ 12.325397] systemd[1]: initrd-switch-root.service: Succeeded. [ 12.330500] systemd[1]: Stopped Switch Root. [ OK ] Stopped Switch Root. [ 12.335872] systemd[1]: systemd-journald.service: Service has no hold-off time (RestartSec=0), scheduling restart. [ 12.341353] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. [ 12.344464] systemd[1]: Stopped Journal Service. [ OK ] Stopped Journal Service. [ 12.351841] systemd[1]: Starting Journal Service... Starting Journal Service... [ 12.356180] systemd[1]: Reached target rpc_pipefs.target. [ OK ] Reached target rpc_pipefs.target. Mounting Huge Pages File System... Mounting POSIX Message Queue File System... [ OK ] Listening on Process Core Dump Socket. [FAILED] Failed to set up automount Arbitrar…rmats File System Automount Point. See 'systemctl status proc-sys-fs-binfmt_misc.automount' for details. Starting Apply Kernel Variables... Starting Create list of required st…ce nodes for the current kernel... [ OK ] Started Forward Password Requests to Wall Directory Watch. Mounting Kernel Debug File System... [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Reached target Local Encrypted Volumes. [ OK ] Reached target Paths. [ OK ] Created slice User and Session Slice. [ OK ] Reached target Slices. [ OK ] Listening on udev Kernel Socket. [ OK ] Stopped target Switch Root. [ OK ] Listening on initctl Compatibility Named Pipe. Activating swap /dev/disk/by-label/SWAP... [ OK ] Created slice system-sshd\x2dkeygen.slice. [ OK ] Stopped target Initrd Root File System. [ OK ] Listening on RPCbind Server Activation Socket. [ OK ] Reached target RPC Port Mapper. [ OK ] Created slice system-getty.slice. [ 12.487450] Adding 1048572k swap on /dev/vdb. Priority:-2 extents:1 across:1048572k FS Starting Remount Root and Kernel File Systems... [ OK ] Stopped target Initrd File Systems. [ OK ] Created slice system-serial\x2dgetty.slice. [ OK ] Listening on udev Control Socket. Starting udev Coldplug all Devices... [ OK ] Started Journal Service. [ OK ] Mounted Huge Pages File System. [ OK ] Mounted POSIX Message Queue File System. [ OK ] Started Apply Kernel Variables. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Mounted Kernel Debug File System. [ OK ] Activated swap /dev/disk/by-label/SWAP. [FAILED] Failed to start Remount Root and Kernel File Systems. See 'systemctl status systemd-remount-fs.service' for details. Starting Configure read-only root support... [ OK ] Reached target Swap. Starting Create Static Device Nodes in /dev... Starting Flush Journal to Persistent Storage... [ OK ] Started Flush Journal to Persistent Storage. [ OK ] Started Create Static Device Nodes in /dev. [ OK ] Reached target Local File Systems (Pre). Mounting /home/green/git/lustre-release... Mounting /mnt... Starting udev Kernel Device Manager... [ OK ] Mounted /mnt. [ 12.779531] squashfs: version 4.0 (2009/01/31) Phillip Lougher [ OK ] Mounted /home/green/git/lustre-release. [ OK ] Started udev Coldplug all Devices. [ OK ] Started udev Kernel Device Manager. [ 13.143621] piix4_smbus 0000:00:01.3: SMBus Host Controller at 0x700, revision 0 [ 13.166290] input: PC Speaker as /devices/platform/pcspkr/input/input5 [ 13.335236] RAPL PMU: API unit is 2^-32 Joules, 0 fixed counters, 10737418240 ms ovfl timer [ 13.350101] EDAC sbridge: Ver: 1.1.2 [ 15.059269] Key type dns_resolver registered [ 15.374342] NFS: Registering the id_resolver key type [ 15.376207] Key type id_resolver registered [ 15.377919] Key type id_legacy registered [ OK ] Started Configure read-only root support. Starting Load/Save Random Seed... [ OK ] Reached target Local File Systems. Starting Mark the need to relabel after reboot... Starting Rebuild Dynamic Linker Cache... Starting Create Volatile Files and Directories... [ OK ] Started Load/Save Random Seed. [ OK ] Started Mark the need to relabel after reboot. [ OK ] Started Create Volatile Files and Directories. Starting RPC Bind... Starting Update UTMP about System Boot/Shutdown... [ OK ] Started Update UTMP about System Boot/Shutdown. [ OK ] Started RPC Bind. [ OK ] Started Rebuild Dynamic Linker Cache. Starting Update is Completed... [ OK ] Started Update is Completed. [ OK ] Reached target System Initialization. [ OK ] Started daily update of the root trust anchor for DNSSEC. [ OK ] Started dnf makecache --timer. [ OK ] Started Daily Cleanup of Temporary Directories. [ OK ] Reached target Timers. [ OK ] Listening on D-Bus System Message Bus Socket. [ OK ] Reached target Sockets. [ OK ] Reached target Basic System. [ OK ] Started D-Bus System Message Bus. Starting Restore /run/initramfs on shutdown... Starting Network Manager... [ OK ] Reached target sshd-keygen.target. [ OK ] Started irqbalance daemon. Starting Login Service... [ OK ] Started Hardware RNG Entropy Gatherer Daemon. [ OK ] Started Restore /run/initramfs on shutdown. [ OK ] Started Login Service. [ OK ] Started Network Manager. Starting Network Manager Wait Online... [ OK ] Reached target Network. Starting OpenSSH server daemon... Starting Dynamic System Tuning Daemon... Starting GSSAPI Proxy Daemon... [ OK ] Started GSSAPI Proxy Daemon. [ OK ] Reached target NFS client services. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Starting Permit User Sessions... Starting Hostname Service... [ OK ] Started OpenSSH server daemon. [ OK ] Started Permit User Sessions. [ OK ] Started Command Scheduler. [ OK ] Started Serial Getty on ttyS1. [ OK ] Started Serial Getty on ttyS0. [ OK ] Started Getty on tty1. [ OK ] Reached target Login Prompts. [ OK ] Started Hostname Service. Starting Network Manager Script Dispatcher Service... [ OK ] Started Network Manager Script Dispatcher Service. [ OK ] Started Network Manager Wait Online. [ OK ] Reached target Network is Online. Starting Crash recovery kernel arming... Starting Notify NFS peers of a restart... Starting System Logging Service... [ OK ] Started Notify NFS peers of a restart. [ OK ] Started System Logging Service. Starting Authorization Manager... [ OK ] Started Authorization Manager. [ OK ] Started Dynamic System Tuning Daemon. [ OK ] Reached target Multi-User System. [ OK ] Reached target Graphical Interface. Starting Update UTMP about System Runlevel Changes... [ OK ] Started Update UTMP about System Runlevel Changes. Rocky Linux 8.10 (Green Obsidian) Kernel 4.18.0rh8.10-debug on an x86_64 oleg141-server login: [ 42.769910] libcfs: loading out-of-tree module taints kernel. [ 42.792380] Key type ._llcrypt registered [ 42.794084] Key type .llcrypt registered [ 42.845604] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_hostid [ 63.703719] hrtimer: interrupt took 7992002 ns [ 65.432436] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing load_modules_local [ 67.837396] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 67.859292] alg: No test for adler32 (adler32-zlib) [ 69.648694] Lustre: Lustre: Build Version: 2.17.50_3_g69d656b [ 70.524236] LNet: Added LNI 192.168.201.141@tcp [8/256/0/180] [ 72.479169] Key type lgssc registered [ 74.831201] Lustre: Echo OBD driver; http://www.lustre.org/ [ 97.468917] ZFS: Loaded module v2.3.2-1, ZFS pool version 5000, ZFS filesystem version 5 [ 99.167789] LDISKFS-fs (vdc): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 111.750191] LDISKFS-fs (vdd): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 121.885879] LDISKFS-fs (vde): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 131.290213] LDISKFS-fs (vdf): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 153.195268] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing load_modules_local [ 171.606659] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 171.770629] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 171.803272] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 173.296208] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 173.353624] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 173.475248] Lustre: lustre-MDT0000: new disk, initializing [ 173.599100] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 173.639320] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 179.977100] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 195.689656] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 195.795410] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 195.912463] Lustre: 6534:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 195.986800] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 195.997038] Lustre: Skipped 1 previous similar message [ 196.218068] Lustre: lustre-MDT0001: new disk, initializing [ 196.345562] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 196.409386] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 196.436017] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 202.203752] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 208.043392] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 219.585960] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 219.665476] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 220.115302] Lustre: lustre-OST0000: new disk, initializing [ 220.119568] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 220.211334] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 221.634341] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 221.663166] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 221.855585] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 228.115128] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 245.966300] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 246.117827] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 246.309242] Lustre: lustre-OST0001: new disk, initializing [ 246.314668] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 246.404451] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 253.692177] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 256.620526] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 256.636327] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 256.771232] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 268.180537] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 280.357862] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 292.822490] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing check_logdir /tmp/testlogs/ [ 298.991426] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing yml_node [ 304.487403] Lustre: DEBUG MARKER: Client: 2.17.50.3 [ 307.548952] Lustre: DEBUG MARKER: MDS: 2.17.50.3 [ 310.546124] Lustre: DEBUG MARKER: OSS: 2.17.50.3 [ 312.405497] Lustre: DEBUG MARKER: -----============= acceptance-small: conf-sanity ============----- Thu Jan 22 04:27:42 EST 2026 [ 333.870385] Lustre: DEBUG MARKER: excepting tests: 102 106 115 32newtarball [ 335.830693] Lustre: DEBUG MARKER: skipping tests SLOW=no: 45 69 106 111 114 [ 354.274082] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 354.280620] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 354.287475] Lustre: Skipped 1 previous similar message [ 354.303351] Lustre: Skipped 3 previous similar messages [ 358.183739] LustreError: 12244:0:(obd_class.h:479:obd_check_dev()) Device 28 not setup [ 358.596459] Lustre: server umount lustre-MDT0000 complete [ 363.118478] LustreError: 6525:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1769074115 with bad export cookie 5828439949872782251 [ 363.136372] LustreError: 6525:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 363.147404] LustreError: MGC192.168.201.141@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 364.511958] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 364.512610] LustreError: 6541:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 364.512757] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 364.523734] Lustre: Skipped 3 previous similar messages [ 364.575145] LustreError: 6541:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 6 previous similar messages [ 368.546100] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 368.549768] LustreError: 6541:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 368.562614] Lustre: Skipped 1 previous similar message [ 368.601821] LustreError: 6541:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 369.431758] LustreError: 12447:0:(obd_class.h:479:obd_check_dev()) Device 27 not setup [ 369.440236] LustreError: 12447:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 370.035927] Lustre: server umount lustre-MDT0001 complete [ 374.758455] LustreError: 12649:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 374.761570] LustreError: 12649:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 374.818990] Lustre: server umount lustre-OST0000 complete [ 379.236862] LustreError: 12850:0:(obd_class.h:479:obd_check_dev()) Device 2 not setup [ 379.240740] LustreError: 12850:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 379.370767] Lustre: server umount lustre-OST0001 complete [ 387.922840] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_hostid [ 398.496962] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing load_modules_local [ 410.580652] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 419.716745] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 427.357954] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 434.974704] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 443.155551] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 443.247462] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 443.507139] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 443.558674] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 443.661929] Lustre: lustre-MDT0000: new disk, initializing [ 443.807840] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 443.872658] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 448.269522] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 461.198819] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 461.238313] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 461.322755] Lustre: 15040:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 461.388616] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 461.392744] Lustre: Skipped 1 previous similar message [ 461.485619] Lustre: lustre-MDT0001: new disk, initializing [ 461.682727] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 461.757313] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 461.781107] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 467.998668] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 477.583681] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 482.805710] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 491.273092] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 491.403667] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 491.885279] Lustre: lustre-OST0000: new disk, initializing [ 491.898281] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 492.076598] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 493.340042] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 493.354893] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 493.649223] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 500.622456] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 510.499084] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 516.051449] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 516.308355] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 521.627442] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 521.873195] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 529.384899] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 529.392667] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 529.421558] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 529.424257] Lustre: Skipped 1 previous similar message [ 530.401109] LustreError: 17982:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 530.403690] LustreError: 17982:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 530.961654] Lustre: server umount lustre-OST0000 complete [ 538.593487] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 538.608817] LustreError: Skipped 1 previous similar message [ 538.617174] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 538.629635] Lustre: Skipped 1 previous similar message [ 538.635074] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 540.897353] LustreError: 18184:0:(obd_class.h:479:obd_check_dev()) Device 22 not setup [ 540.906086] LustreError: 18184:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 541.026982] Lustre: server umount lustre-MDT0000 complete [ 544.736566] LustreError: 15050:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 544.773597] LustreError: 15050:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 545.452036] LustreError: 15820:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1769074297 with bad export cookie 5828439949872783784 [ 545.459365] LustreError: MGC192.168.201.141@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 545.465762] LustreError: 15820:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 545.945354] Lustre: server umount lustre-MDT0001 complete [ 552.057360] Lustre: DEBUG MARKER: == conf-sanity test 76a: set permanent params with lctl across mounts ========================================================== 04:31:42 (1769074302) [ 559.662981] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 560.303466] LustreError: 18912:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 560.437904] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 565.736888] LustreError: 18912:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 565.793442] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 575.597755] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 575.981384] Lustre: lustre-MDT0001: Not available for connect from 0@lo (not set up) [ 575.992596] Lustre: Skipped 2 previous similar messages [ 581.561816] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 587.604258] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 592.149377] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 600.639820] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 607.785616] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 614.969631] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 619.047838] Lustre: Modifying parameter general.osc.*.max_dirty_mb=452 in log params [ 625.634766] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 625.638274] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 625.645331] Lustre: Skipped 2 previous similar messages [ 630.875688] LustreError: 21214:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup [ 630.889015] LustreError: 21214:0:(obd_class.h:479:obd_check_dev()) Skipped 13 previous similar messages [ 631.235855] Lustre: server umount lustre-MDT0000 complete [ 635.060455] LustreError: 21108:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1769074387 with bad export cookie 5828439949872784666 [ 635.064237] LustreError: MGC192.168.201.141@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 635.081481] LustreError: 21108:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 1 previous similar message [ 635.839770] Lustre: server umount lustre-MDT0001 complete [ 650.398032] Lustre: server umount lustre-OST0000 complete [ 676.139584] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing load_modules_local [ 688.299595] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 688.934742] LustreError: 22948:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 688.960648] LustreError: 22948:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 689.077311] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 689.087969] Lustre: Skipped 2 previous similar messages [ 694.339572] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 704.850585] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 710.468106] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 714.382969] Lustre: Setting parameter general.debug_raw_pointers=Y in log params [ 722.370452] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 730.094761] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 739.286702] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 739.456523] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 739.716474] Lustre: lustre-OST0001: new disk, initializing [ 739.721670] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 741.580098] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 741.590268] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 741.691892] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 747.398759] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 758.607295] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 762.815715] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 781.796438] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 781.811927] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 781.831769] Lustre: Skipped 1 previous similar message [ 781.836778] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 781.841354] Lustre: Skipped 5 previous similar messages [ 787.142236] LustreError: 26435:0:(obd_class.h:479:obd_check_dev()) Device 28 not setup [ 787.153244] LustreError: 26435:0:(obd_class.h:479:obd_check_dev()) Skipped 17 previous similar messages [ 787.287877] Lustre: server umount lustre-MDT0000 complete [ 787.940707] LustreError: 22944:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 787.960260] LustreError: 22944:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 7 previous similar messages [ 791.423956] LustreError: 22929:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1769074543 with bad export cookie 5828439949872785513 [ 791.449471] LustreError: MGC192.168.201.141@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 792.034444] Lustre: server umount lustre-MDT0001 complete [ 806.953714] Lustre: server umount lustre-OST0000 complete [ 808.936672] Lustre: 3669:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1769074545/real 1769074545] req@ffff9e1281858700 x1855008352239104/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1769074561 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 809.001230] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 809.018440] Lustre: Skipped 3 previous similar messages [ 811.905732] Lustre: server umount lustre-OST0001 complete [ 829.401440] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing load_modules_local [ 841.299526] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 842.092126] LustreError: 28215:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 842.191431] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 842.197786] Lustre: Skipped 3 previous similar messages [ 847.731682] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 858.013128] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 864.553290] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 867.547873] Lustre: 29332:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 876.735224] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 884.426332] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 895.184369] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 902.248528] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 910.359394] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 914.593759] Lustre: 31190:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 930.277187] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 930.279979] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 930.312505] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 930.318939] Lustre: Skipped 4 previous similar messages [ 932.479571] LustreError: 31392:0:(obd_class.h:479:obd_check_dev()) Device 10 not setup [ 932.491037] LustreError: 31392:0:(obd_class.h:479:obd_check_dev()) Skipped 25 previous similar messages [ 932.859287] Lustre: server umount lustre-MDT0000 complete [ 936.427231] LustreError: 31394:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 936.447146] LustreError: 31394:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 11 previous similar messages [ 937.115376] LustreError: 28195:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1769074689 with bad export cookie 5828439949872786801 [ 937.119819] LustreError: MGC192.168.201.141@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 937.125798] LustreError: 28195:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 6 previous similar messages [ 937.402466] Lustre: server umount lustre-MDT0001 complete [ 951.970149] Lustre: server umount lustre-OST0000 complete [ 965.257993] Lustre: server umount lustre-OST0001 complete [ 974.238849] Lustre: DEBUG MARKER: == conf-sanity test 76b: verify params log setup correctly ========================================================== 04:38:45 (1769074725) [ 1004.102611] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing load_modules_local [ 1015.292135] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1020.993291] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 1030.266243] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1035.586087] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 1038.632943] Lustre: 35096:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1045.337883] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1052.457502] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 1062.986274] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1070.758772] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 1078.321622] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1081.740176] Lustre: 36950:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 1092.067748] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1092.070430] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1092.086198] Lustre: Skipped 3 previous similar messages [ 1092.091091] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1092.095576] Lustre: Skipped 4 previous similar messages [ 1098.201850] LustreError: 37102:0:(obd_class.h:479:obd_check_dev()) Device 10 not setup [ 1098.206701] LustreError: 37102:0:(obd_class.h:479:obd_check_dev()) Skipped 25 previous similar messages [ 1098.707258] Lustre: server umount lustre-MDT0000 complete [ 1099.241686] LustreError: 33980:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1099.265840] LustreError: 33980:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 9 previous similar messages [ 1102.920113] LustreError: 33961:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1769074855 with bad export cookie 5828439949872787914 [ 1102.921532] LustreError: MGC192.168.201.141@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1102.933511] LustreError: 33961:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 1103.351229] Lustre: server umount lustre-MDT0001 complete [ 1117.153115] Lustre: server umount lustre-OST0000 complete [ 1130.731543] Lustre: server umount lustre-OST0001 complete [ 1141.571890] Lustre: DEBUG MARKER: == conf-sanity test 76c: verify changelog_mask is applied with lctl set_param -P ========================================================== 04:41:32 (1769074892) [ 1156.178641] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing load_modules_local [ 1169.293644] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1170.260445] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1170.270906] Lustre: Skipped 7 previous similar messages [ 1176.164335] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 1185.469277] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1191.304722] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 1194.103877] Lustre: 40191:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1200.973397] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1207.271266] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 1215.332816] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1221.971365] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 1229.442284] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1232.928552] Lustre: 42047:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 1239.136914] Lustre: Setting parameter general.mdd.*.changelog_mask=-CLOSE in log params [ 1239.142518] Lustre: Skipped 1 previous similar message [ 1242.592327] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1242.602343] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1242.620651] Lustre: Skipped 6 previous similar messages [ 1248.883808] Lustre: server umount lustre-MDT0000 complete [ 1253.115709] LustreError: 39057:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1769075005 with bad export cookie 5828439949872789020 [ 1253.118808] LustreError: MGC192.168.201.141@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1253.135806] LustreError: 39057:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1259.577163] Lustre: server umount lustre-MDT0001 complete [ 1268.057167] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1278.434343] LustreError: 3668:0:(client.c:1390:ptlrpc_import_delay_req()) @@@ invalidate in flight req@ffff9e11486e5500 x1855008352404352/t0(0) o250->MGC192.168.201.141@tcp@0@lo:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl Rpc:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1284.116747] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 1288.366308] Lustre: lustre-MDT0000-lwp-OST0001: Connection restored to 0@lo (at 0@lo) [ 1292.704881] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1298.216764] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 1298.421803] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 1298.429680] Lustre: Skipped 1 previous similar message [ 1304.319710] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1306.170486] Lustre: DEBUG MARKER: mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec [ 1309.868191] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 1311.304493] Lustre: DEBUG MARKER: mdc.lustre-MDT0001-mdc-*.mds_server_uuid in FULL state after 0 sec [ 1318.904097] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1325.197239] Lustre: server umount lustre-MDT0000 complete [ 1329.599817] LustreError: 39059:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1769075081 with bad export cookie 5828439949872790210 [ 1329.611075] LustreError: MGC192.168.201.141@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1329.617587] LustreError: 39059:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 1330.008610] Lustre: server umount lustre-MDT0001 complete [ 1344.462799] Lustre: server umount lustre-OST0000 complete [ 1358.740874] LustreError: 45053:0:(obd_class.h:479:obd_check_dev()) Device 2 not setup [ 1358.749928] LustreError: 45053:0:(obd_class.h:479:obd_check_dev()) Skipped 61 previous similar messages [ 1359.223894] Lustre: server umount lustre-OST0001 complete [ 1368.926852] Lustre: DEBUG MARKER: == conf-sanity test 76d: verify llite.*.xattr_cache can be set by 'lctl set_param -P' correctly ========================================================== 04:45:19 (1769075119) [ 1387.402672] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing load_modules_local [ 1398.329976] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1398.871043] LustreError: 46426:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1398.894087] LustreError: 46426:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 25 previous similar messages [ 1403.754829] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 1412.160914] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1418.391296] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 1421.935754] Lustre: 47550:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1429.929527] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1438.093081] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 1446.474517] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1452.828676] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 1462.009518] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1471.912619] Lustre: Setting parameter general.llite.*.xattr_cache=0 in log params [ 1484.256536] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1484.264720] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1484.271273] Lustre: Skipped 27 previous similar messages [ 1489.453964] Lustre: server umount lustre-MDT0000 complete [ 1493.077215] LustreError: 46405:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1769075245 with bad export cookie 5828439949872790994 [ 1493.084546] LustreError: MGC192.168.201.141@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1493.085198] LustreError: 46405:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1493.377556] Lustre: server umount lustre-MDT0001 complete [ 1507.466057] Lustre: server umount lustre-OST0000 complete [ 1522.180091] Lustre: server umount lustre-OST0001 complete [ 1531.951419] Lustre: DEBUG MARKER: == conf-sanity test 77: comma-separated MGS NIDs and failover node NIDs ========================================================== 04:48:02 (1769075282) [ 1534.728915] Lustre: DEBUG MARKER: SKIP: conf-sanity test_77 mixed loopback and real device not working [ 1537.284359] Lustre: DEBUG MARKER: == conf-sanity test 78: run resize2fs on MDT and OST filesystems ========================================================== 04:48:07 (1769075287) [ 1549.500533] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_hostid [ 1559.624911] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing load_modules_local [ 1570.992060] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1577.540929] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1585.109558] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1585.178352] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1585.396558] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1585.521256] Lustre: lustre-MDT0000: new disk, initializing [ 1585.659502] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1590.390704] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 1597.682522] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1604.292061] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1604.404497] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1604.704829] Lustre: lustre-OST0000: new disk, initializing [ 1604.712400] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1604.721449] Lustre: Skipped 1 previous similar message [ 1606.586950] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 1606.595155] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 1606.632979] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 1612.185294] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 1621.912748] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1624.615575] Lustre: DEBUG MARKER: create test files [ 1662.944291] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1662.955798] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1662.964687] Lustre: Skipped 10 previous similar messages [ 1667.547856] Lustre: server umount lustre-OST0000 complete [ 1672.633096] Lustre: server umount lustre-MDT0000 complete [ 1684.425334] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing unload_modules_local [ 1688.532276] Key type lgssc unregistered [ 1688.948770] LNet: 54974:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1688.969940] LNetError: 54974:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1690.024957] LNet: Removed LNI 192.168.201.141@tcp [ 1691.403149] Key type .llcrypt unregistered [ 1691.405394] Key type ._llcrypt unregistered [ 1725.253312] Key type ._llcrypt registered [ 1725.255255] Key type .llcrypt registered [ 1725.347868] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing load_modules_local [ 1726.598068] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1726.642612] alg: No test for adler32 (adler32-zlib) [ 1727.621079] Lustre: Lustre: Build Version: 2.17.50_3_g69d656b [ 1727.809666] LNet: Added LNI 192.168.201.141@tcp [8/256/0/180] [ 1729.487250] Key type lgssc registered [ 1731.044595] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1741.657581] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1741.684223] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1743.344995] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1748.394821] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 1754.928478] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1762.686322] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1763.123397] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1768.234945] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:102 to 0x240000400:129) [ 1768.410615] Lustre: 57587:0:(osd_internal.h:1459:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 515 < left 544, rollback = 9 [ 1768.423483] Lustre: 57587:0:(osd_handler.c:2096:osd_trans_dump_creds()) create: 32/128/0, destroy: 0/0/0 [ 1768.449828] Lustre: 57587:0:(osd_handler.c:2103:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 1768.468205] Lustre: 57587:0:(osd_handler.c:2113:osd_trans_dump_creds()) write: 2/13/1, punch: 0/0/0, quota 1/3/0 [ 1768.474103] Lustre: 57587:0:(osd_handler.c:2120:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 1768.480042] Lustre: 57587:0:(osd_handler.c:2127:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 1770.216587] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 1776.948603] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1782.560780] Lustre: DEBUG MARKER: check files after expanding the MDT and OST filesystems [ 1788.125311] Lustre: DEBUG MARKER: create more files after expanding the MDT and OST filesystems [ 1824.741262] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1824.745472] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1824.754993] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1827.265579] LustreError: 58356:0:(obd_class.h:479:obd_check_dev()) Device 13 not setup [ 1827.464826] Lustre: server umount lustre-OST0000 complete [ 1831.912825] LustreError: 58557:0:(obd_class.h:479:obd_check_dev()) Device 8 not setup [ 1831.923830] LustreError: 58557:0:(obd_class.h:479:obd_check_dev()) Skipped 1 previous similar message [ 1832.273988] Lustre: server umount lustre-MDT0000 complete [ 1845.340370] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing unload_modules_local [ 1849.033934] Key type lgssc unregistered [ 1849.579913] LNet: 59090:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1849.597107] LNetError: 59090:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1849.631707] LNet: Removed LNI 192.168.201.141@tcp [ 1850.759723] Key type .llcrypt unregistered [ 1850.764418] Key type ._llcrypt unregistered [ 1887.929565] Key type ._llcrypt registered [ 1887.932892] Key type .llcrypt registered [ 1888.089625] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing load_modules_local [ 1889.880702] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1889.954148] alg: No test for adler32 (adler32-zlib) [ 1891.254751] Lustre: Lustre: Build Version: 2.17.50_3_g69d656b [ 1891.553215] LNet: Added LNI 192.168.201.141@tcp [8/256/0/180] [ 1893.337209] Key type lgssc registered [ 1894.892178] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1905.252557] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1905.276369] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1907.103577] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1912.601857] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 1919.483961] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1928.127828] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1928.812604] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1932.924420] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:140 to 0x240000400:161) [ 1933.067401] Lustre: 61802:0:(osd_internal.h:1459:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 515 < left 544, rollback = 9 [ 1933.080606] Lustre: 61802:0:(osd_handler.c:2096:osd_trans_dump_creds()) create: 32/128/0, destroy: 0/0/0 [ 1933.089060] Lustre: 61802:0:(osd_handler.c:2103:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 1933.098462] Lustre: 61802:0:(osd_handler.c:2113:osd_trans_dump_creds()) write: 2/13/1, punch: 0/0/0, quota 1/3/0 [ 1933.113067] Lustre: 61802:0:(osd_handler.c:2120:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 1933.122313] Lustre: 61802:0:(osd_handler.c:2127:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 1935.942867] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 1942.540957] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1949.148725] Lustre: DEBUG MARKER: check files after shrinking the MDT and OST filesystems [ 1958.900748] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1958.924184] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1958.928624] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1963.469192] LustreError: 62496:0:(obd_class.h:479:obd_check_dev()) Device 13 not setup [ 1963.707160] Lustre: server umount lustre-OST0000 complete [ 1968.916303] LustreError: 62697:0:(obd_class.h:479:obd_check_dev()) Device 8 not setup [ 1968.927639] LustreError: 62697:0:(obd_class.h:479:obd_check_dev()) Skipped 1 previous similar message [ 1969.213795] Lustre: server umount lustre-MDT0000 complete [ 1982.621268] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing unload_modules_local [ 1986.199605] Key type lgssc unregistered [ 1986.542931] LNet: 63231:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1986.560394] LNetError: 63231:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1986.575393] LNet: Removed LNI 192.168.201.141@tcp [ 1987.573080] Key type .llcrypt unregistered [ 1987.578121] Key type ._llcrypt unregistered [ 2010.587525] Key type ._llcrypt registered [ 2010.589759] Key type .llcrypt registered [ 2010.731468] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_hostid [ 2030.819861] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing load_modules_local [ 2032.293745] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2032.407155] alg: No test for adler32 (adler32-zlib) [ 2033.650297] Lustre: Lustre: Build Version: 2.17.50_3_g69d656b [ 2034.040599] LNet: Added LNI 192.168.201.141@tcp [8/256/0/180] [ 2035.719588] Key type lgssc registered [ 2037.411977] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2048.191443] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2055.791956] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2062.372025] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2069.561585] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2076.753127] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2076.795722] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 2076.809616] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2078.107750] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2078.144197] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2078.231792] Lustre: lustre-MDT0000: new disk, initializing [ 2078.371414] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2078.391233] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2083.217547] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 2094.767513] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2094.828369] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2094.910260] Lustre: 66365:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2094.983184] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 2094.987365] Lustre: Skipped 1 previous similar message [ 2095.068610] Lustre: lustre-MDT0001: new disk, initializing [ 2095.166209] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2095.199240] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 2095.209674] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2100.576895] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 2109.439973] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2113.561871] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2121.290144] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2121.415444] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2121.786286] Lustre: lustre-OST0000: new disk, initializing [ 2121.798484] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 2121.944884] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2128.519073] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 2128.550207] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2128.787721] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2129.092833] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 2138.411844] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2143.638709] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2143.882963] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2150.112628] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2150.316950] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2154.467275] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2154.470399] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2154.474307] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2154.518729] Lustre: Skipped 1 previous similar message [ 2154.559230] Lustre: Skipped 1 previous similar message [ 2158.539170] LustreError: 69304:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 2158.769782] Lustre: server umount lustre-OST0000 complete [ 2164.708680] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2164.723113] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2164.729382] Lustre: Skipped 1 previous similar message [ 2164.746236] Lustre: Skipped 1 previous similar message [ 2169.237330] LustreError: 69506:0:(obd_class.h:479:obd_check_dev()) Device 22 not setup [ 2169.242326] LustreError: 69506:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 2169.304626] Lustre: server umount lustre-MDT0000 complete [ 2173.417504] LustreError: 66355:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1769075925 with bad export cookie 5192416805444436956 [ 2173.422612] LustreError: MGC192.168.201.141@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2173.435508] LustreError: 66355:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2173.647586] LustreError: 69708:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 2173.656953] LustreError: 69708:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 2174.065717] Lustre: server umount lustre-MDT0001 complete [ 2183.049684] Lustre: DEBUG MARKER: == conf-sanity test 79: format MDT/OST without mgs option (should return errors) ========================================================== 04:58:53 (1769075933) [ 2195.034950] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing load_modules_local [ 2209.692987] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2239.307390] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_hostid [ 2251.284388] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing load_modules_local [ 2262.880094] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2270.358171] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2277.750529] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2284.157823] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2291.532672] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2291.608616] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2291.841577] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2291.876781] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2291.975209] Lustre: lustre-MDT0000: new disk, initializing [ 2292.067563] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2292.082715] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2296.625268] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 2307.281420] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2307.389708] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2307.522163] Lustre: 73949:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2307.649029] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 2307.655392] Lustre: Skipped 1 previous similar message [ 2307.788484] Lustre: lustre-MDT0001: new disk, initializing [ 2307.960396] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2308.054639] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 2308.079027] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2313.791471] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 2322.009628] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2325.833544] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2333.656167] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2333.735048] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2334.002817] Lustre: lustre-OST0000: new disk, initializing [ 2334.007630] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 2334.095951] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2341.036956] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 2343.517228] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 2343.523099] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2343.601216] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2348.687714] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2353.905763] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2354.148593] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2359.005577] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2359.199739] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2364.384312] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2364.385887] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2364.397107] LustreError: Skipped 2 previous similar messages [ 2364.397977] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2364.415147] Lustre: Skipped 1 previous similar message [ 2364.448331] Lustre: Skipped 1 previous similar message [ 2369.511916] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2369.516884] Lustre: Skipped 1 previous similar message [ 2374.628468] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2374.635129] Lustre: Skipped 1 previous similar message [ 2375.135523] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2375.197759] LustreError: 76907:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 2375.201470] LustreError: 76907:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 2375.305743] Lustre: server umount lustre-OST0000 complete [ 2379.745066] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2379.752367] Lustre: Skipped 1 previous similar message [ 2384.529845] LustreError: 77110:0:(obd_class.h:479:obd_check_dev()) Device 22 not setup [ 2384.535676] LustreError: 77110:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 2384.644243] Lustre: server umount lustre-MDT0000 complete [ 2384.874798] LustreError: 73961:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2384.904099] LustreError: 73961:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 4 previous similar messages [ 2388.957558] LustreError: 73941:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1769076141 with bad export cookie 5192416805444437838 [ 2388.965686] LustreError: MGC192.168.201.141@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2388.970733] LustreError: 73941:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 2389.497776] Lustre: server umount lustre-MDT0001 complete [ 2399.055870] Lustre: DEBUG MARKER: == conf-sanity test 80: mgc import reconnect race ======== 05:02:29 (1769076149) [ 2406.227029] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2406.994080] LustreError: 77982:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2407.086675] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2412.191932] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 2412.515162] LustreError: 77981:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2412.529592] LustreError: 77981:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 2417.637231] LustreError: 77981:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2421.565927] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2427.253474] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 2433.592881] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2438.051993] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2445.915076] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2446.395620] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2446.405339] Lustre: Skipped 1 previous similar message [ 2453.900492] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 2459.392565] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2464.083484] Lustre: 80322:0:(genops.c:1791:obd_export_evict_by_uuid()) MGS: evicting 009a2edd-b73b-4396-8dbe-a590c028f6b7 at adminstrative request [ 2467.821729] LustreError: MGC192.168.201.141@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2467.846857] Lustre: *** cfs_fail_loc=906, val=2147483648*** [ 2477.030590] Lustre: MGS: Client 009a2edd-b73b-4396-8dbe-a590c028f6b7 (at 0@lo) reconnecting [ 2477.042286] Lustre: Evicted from MGS (at 0@lo) after server handle changed from 0x480f2b92ee48cab9 to 0x480f2b92ee48cd60 [ 2477.062600] Lustre: MGC192.168.201.141@tcp: Connection restored to 0@lo (at 0@lo) [ 2503.495462] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2503.546747] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2503.667702] Lustre: lustre-OST0001: new disk, initializing [ 2503.679594] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 2510.522192] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 2510.538400] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 2510.633773] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 2512.815605] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 2521.367956] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid [ 2526.190724] LustreError: lustre-OST0001-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2526.193045] Lustre: lustre-OST0001-osc-MDT0001: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2526.205212] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 2526.205221] Lustre: Skipped 2 previous similar messages [ 2526.205938] LustreError: Skipped 1 previous similar message [ 2526.261306] Lustre: Skipped 1 previous similar message [ 2530.049708] LustreError: 81461:0:(obd_class.h:479:obd_check_dev()) Device 25 not setup [ 2530.054124] LustreError: 81461:0:(obd_class.h:479:obd_check_dev()) Skipped 13 previous similar messages [ 2530.221537] Lustre: server umount lustre-OST0001 complete [ 2531.297916] LustreError: 79583:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2531.317662] LustreError: 79583:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 2534.370200] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2534.378699] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2534.399954] Lustre: Skipped 1 previous similar message [ 2540.319517] Lustre: server umount lustre-OST0000 complete [ 2545.121769] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2545.133709] LustreError: Skipped 1 previous similar message [ 2545.143535] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2545.150621] Lustre: Skipped 5 previous similar messages [ 2550.644701] LustreError: 81866:0:(obd_class.h:479:obd_check_dev()) Device 28 not setup [ 2550.662606] LustreError: 81866:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 2551.005620] Lustre: server umount lustre-MDT0000 complete [ 2551.778739] LustreError: 81172:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2551.791372] LustreError: 81172:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 2555.176930] LustreError: 80727:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1769076307 with bad export cookie 5192416805444439392 [ 2555.198401] LustreError: 80727:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 2555.202263] LustreError: MGC192.168.201.141@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2555.880404] Lustre: server umount lustre-MDT0001 complete [ 2566.165375] Lustre: DEBUG MARKER: == conf-sanity test 81: sparse OST indexing ============== 05:05:16 (1769076316) [ 2568.175140] Lustre: DEBUG MARKER: SKIP: conf-sanity test_81 needs >= 3 OSTs [ 2570.490852] Lustre: DEBUG MARKER: == conf-sanity test 82a: specify OSTs for file (succeed) or directory (succeed) ========================================================== 05:05:20 (1769076320) [ 2572.210141] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82a needs >= 3 OSTs [ 2574.530045] Lustre: DEBUG MARKER: == conf-sanity test 82b: specify OSTs for file with --pool and --ost-list options ========================================================== 05:05:24 (1769076324) [ 2576.499493] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82b needs >= 4 OSTs [ 2578.858215] Lustre: DEBUG MARKER: == conf-sanity test 82c: specify sparse OSTs for setstripe (should not crash) ========================================================== 05:05:28 (1769076328) [ 2580.486905] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82c needs >= 4 OSTs [ 2582.739809] Lustre: DEBUG MARKER: == conf-sanity test 83: ENOSPACE on OST doesn't cause message VFS: Busy inodes after unmount ... ========================================================== 05:05:32 (1769076332) [ 2585.026845] Lustre: DEBUG MARKER: mount the OST /dev/mapper/ost1_flakey as a ldiskfs filesystem [ 2590.529986] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2591.830200] loop: module loaded [ 2591.860533] loop0: detected capacity change from 0 to 4294967296 [ 2591.923594] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 2593.978753] Lustre: DEBUG MARKER: run llverfs in partial mode on the OST ldiskfs /mnt/lustre-ost1 [ 2598.701851] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing run_llverfs /mnt/lustre-ost1 -vpl no [ 2604.507549] Lustre: DEBUG MARKER: unmount the OST /dev/mapper/ost1_flakey [ 2614.288839] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2619.349434] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2623.879956] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2628.558968] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2636.182346] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2636.307551] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2636.336621] LustreError: 84742:0:(obd_config.c:783:class_setup()) setup lustre-OST0000-osd failed (-28) [ 2636.351434] LustreError: 84742:0:(obd_mount.c:193:lustre_start_simple()) lustre-OST0000-osd setup error -28 [ 2636.356821] LustreError: 84742:0:(tgt_mount.c:2447:server_fill_super()) Unable to start osd on /dev/mapper/ost1_flakey: -28 [ 2636.366499] LustreError: 84742:0:(super25.c:199:lustre_fill_super()) llite: Unable to mount : rc = -28 [ 2661.046284] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_hostid [ 2674.720462] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing load_modules_local [ 2686.705133] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2694.170150] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2701.860031] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2710.446372] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2719.121691] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2719.208136] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2719.475811] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2719.538303] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2719.663630] Lustre: lustre-MDT0000: new disk, initializing [ 2719.841644] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2719.848571] Lustre: Skipped 1 previous similar message [ 2719.887185] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2725.663942] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 2737.960323] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2738.039685] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2738.163140] Lustre: 87603:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2738.619914] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2744.312000] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 2752.717115] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2757.213964] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2767.180342] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2767.317407] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2775.850788] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 2777.235679] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2777.417339] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2785.882287] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2791.895879] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2792.195874] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2798.616407] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2798.947871] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2803.183817] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2803.184067] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2803.191197] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2803.191206] Lustre: Skipped 2 previous similar messages [ 2803.210703] Lustre: Skipped 3 previous similar messages [ 2807.375585] LustreError: 90549:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 2807.386062] LustreError: 90549:0:(obd_class.h:479:obd_check_dev()) Skipped 17 previous similar messages [ 2807.557667] Lustre: server umount lustre-OST0000 complete [ 2818.945718] Lustre: server umount lustre-MDT0000 complete [ 2823.648560] LustreError: 87612:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2823.678230] LustreError: 87612:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 2823.835469] LustreError: 87597:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1769076575 with bad export cookie 5192416805444439987 [ 2823.856885] LustreError: MGC192.168.201.141@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2823.859734] LustreError: 87597:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2824.363945] Lustre: server umount lustre-MDT0001 complete [ 2835.001381] Lustre: DEBUG MARKER: == conf-sanity test 84: check recovery_hard_time ========= 05:09:45 (1769076585) [ 2843.895471] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2849.533300] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 2859.541347] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2860.227312] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-60 [ 2860.229572] Lustre: Skipped 3 previous similar messages [ 2865.615907] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 2872.672434] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2877.923751] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2888.866707] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2889.133160] LustreError: 93225:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2889.160937] LustreError: 93225:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 2897.550704] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 2904.134323] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2908.782358] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2908.947347] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2914.014209] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2914.299818] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2922.194567] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2922.270417] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2922.393224] Lustre: lustre-OST0001: new disk, initializing [ 2922.395602] Lustre: Skipped 2 previous similar messages [ 2922.405852] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 2922.410038] Lustre: Skipped 3 previous similar messages [ 2930.319900] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 2932.873651] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 2932.880699] Lustre: Skipped 2 previous similar messages [ 2932.887699] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 2933.012493] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 2938.555649] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid [ 2943.819606] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid 50 [ 2944.086685] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2949.716381] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [ 2950.024088] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2970.944311] Lustre: DEBUG MARKER: mds1 REPLAY BARRIER on lustre-MDT0000 [ 2997.932201] Lustre: Failing over lustre-MDT0000 [ 2998.240044] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2998.251191] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2998.270815] Lustre: Skipped 2 previous similar messages [ 2998.282491] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2998.297470] Lustre: Skipped 6 previous similar messages [ 2998.420290] LustreError: 96538:0:(obd_class.h:479:obd_check_dev()) Device 28 not setup [ 2998.425423] LustreError: 96538:0:(obd_class.h:479:obd_check_dev()) Skipped 17 previous similar messages [ 2998.553406] Lustre: server umount lustre-MDT0000 complete [ 2998.814064] LustreError: 95073:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 192.168.201.41@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2998.844960] LustreError: 95073:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 3015.135469] Lustre: 64573:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1769076751/real 1769076751] req@0000000054165b42 x1855010411659520/t0(0) o400->MGC192.168.201.141@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1769076767 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 3015.175265] LustreError: MGC192.168.201.141@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3025.618661] LDISKFS-fs (dm-0): 4 truncates cleaned up [ 3025.623979] LDISKFS-fs (dm-0): recovery complete [ 3025.635241] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3041.335274] Lustre: lustre-MDT0000: in recovery but waiting for the first client to connect [ 3044.810633] Lustre: lustre-MDT0000: Will be in recovery for at least 1:00, or until 3 clients reconnect [ 3044.832728] Lustre: 92576:0:(ldlm_lib.c:2068:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 3046.206416] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 3046.400618] Lustre: 91624:0:(ldlm_lib.c:2068:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 3046.419316] Lustre: lustre-MDT0000-lwp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 3046.425629] Lustre: 91624:0:(ldlm_lib.c:2068:extend_recovery_timer()) Skipped 1 previous similar message [ 3046.580942] LustreError: 97288:0:(ldlm_lib.c:2681:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 3046.914454] LustreError: 97288:0:(ldlm_lib.c:2681:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 3047.251727] LustreError: 97288:0:(ldlm_lib.c:2681:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 3047.257102] LustreError: 97288:0:(ldlm_lib.c:2681:replay_request_or_update()) Skipped 1 previous similar message [ 3047.617124] LustreError: 97288:0:(ldlm_lib.c:2681:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 3047.640396] LustreError: 97288:0:(ldlm_lib.c:2681:replay_request_or_update()) Skipped 1 previous similar message [ 3047.666337] Lustre: 97288:0:(ldlm_lib.c:2068:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 3047.694793] Lustre: 97288:0:(ldlm_lib.c:2068:extend_recovery_timer()) Skipped 7 previous similar messages [ 3048.357350] LustreError: 97288:0:(ldlm_lib.c:2681:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 3048.384939] LustreError: 97288:0:(ldlm_lib.c:2681:replay_request_or_update()) Skipped 2 previous similar messages [ 3048.711127] LustreError: 97288:0:(ldlm_lib.c:2681:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 3048.722070] LustreError: 97288:0:(ldlm_lib.c:2681:replay_request_or_update()) Skipped 2 previous similar messages [ 3049.697449] Lustre: 97288:0:(ldlm_lib.c:2068:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 3049.717292] Lustre: 97288:0:(ldlm_lib.c:2068:extend_recovery_timer()) Skipped 5 previous similar messages [ 3050.356623] LustreError: 97288:0:(ldlm_lib.c:2681:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 3050.371875] LustreError: 97288:0:(ldlm_lib.c:2681:replay_request_or_update()) Skipped 5 previous similar messages [ 3051.037866] LustreError: 97288:0:(ldlm_lib.c:2681:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 3051.053641] LustreError: 97288:0:(ldlm_lib.c:2681:replay_request_or_update()) Skipped 6 previous similar messages [ 3053.970558] Lustre: 97288:0:(ldlm_lib.c:2068:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 3053.990076] Lustre: 97288:0:(ldlm_lib.c:2068:extend_recovery_timer()) Skipped 12 previous similar messages [ 3054.656323] LustreError: 97288:0:(ldlm_lib.c:2681:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 3054.680905] LustreError: 97288:0:(ldlm_lib.c:2681:replay_request_or_update()) Skipped 12 previous similar messages [ 3055.327155] LustreError: 97288:0:(ldlm_lib.c:2681:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 3055.352404] LustreError: 97288:0:(ldlm_lib.c:2681:replay_request_or_update()) Skipped 12 previous similar messages [ 3062.080507] Lustre: 97288:0:(ldlm_lib.c:2068:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 3062.112837] Lustre: 97288:0:(ldlm_lib.c:2068:extend_recovery_timer()) Skipped 24 previous similar messages [ 3062.763536] LustreError: 97288:0:(ldlm_lib.c:2681:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 3062.774856] LustreError: 97288:0:(ldlm_lib.c:2681:replay_request_or_update()) Skipped 24 previous similar messages [ 3063.439710] LustreError: 97288:0:(ldlm_lib.c:2681:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 3063.457780] LustreError: 97288:0:(ldlm_lib.c:2681:replay_request_or_update()) Skipped 24 previous similar messages [ 3078.112515] Lustre: 97288:0:(ldlm_lib.c:2068:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 3078.127133] Lustre: 97288:0:(ldlm_lib.c:2068:extend_recovery_timer()) Skipped 49 previous similar messages [ 3078.784453] LustreError: 97288:0:(ldlm_lib.c:2681:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 3078.807600] LustreError: 97288:0:(ldlm_lib.c:2681:replay_request_or_update()) Skipped 49 previous similar messages [ 3079.457361] LustreError: 97288:0:(ldlm_lib.c:2681:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 3079.476478] LustreError: 97288:0:(ldlm_lib.c:2681:replay_request_or_update()) Skipped 49 previous similar messages [ 3104.192983] Lustre: 97288:0:(ldlm_lib.c:2377:target_recovery_overseer()) lustre-MDT0000 recovery is aborted by hard timeout [ 3104.206332] Lustre: 97288:0:(ldlm_lib.c:2387:target_recovery_overseer()) recovery is aborted, evict exports in recovery [ 3104.229158] Lustre: 97288:0:(genops.c:1618:class_disconnect_stale_exports()) lustre-MDT0000: disconnect stale client f1558f39-b87a-4151-a1c4-36541c8839e9@192.168.201.41@tcp [ 3104.257068] Lustre: lustre-MDT0000: disconnecting 1 stale clients [ 3104.268225] Lustre: 97288:0:(ldlm_lib.c:1897:abort_req_replay_queue()) @@@ aborted: req@00000000da66de40 x1855010390480256/t0(8589934776) o35->f1558f39-b87a-4151-a1c4-36541c8839e9@192.168.201.41@tcp:126/0 lens 392/0 e 0 to 0 dl 1769076866 ref 1 fl Complete:/604/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:0 [ 3104.314444] Lustre: lustre-MDT0000-osd: cancel update llog [0x200000400:0x1:0x0] [ 3104.341862] Lustre: lustre-MDT0000: Denying connection for new client f1558f39-b87a-4151-a1c4-36541c8839e9 (at 192.168.201.41@tcp), waiting for 3 known clients (2 recovered, 0 in progress, and 1 evicted) to recover in 0:00 [ 3104.370535] Lustre: lustre-MDT0001-osp-MDT0000: cancel update llog [0x240000401:0x1:0x0] [ 3104.467736] Lustre: lustre-MDT0000: Recovery over after 1:00, of 3 clients 2 recovered and 1 was evicted. [ 3104.471677] Lustre: lustre-MDT0000-osp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 3104.497649] Lustre: Skipped 2 previous similar messages [ 3104.549152] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:47 to 0x2c0000401:545) [ 3104.555553] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:79 to 0x280000401:577) [ 3123.183162] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3123.208916] Lustre: Skipped 4 previous similar messages [ 3127.808868] LustreError: 97951:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 3127.818892] LustreError: 97951:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 3128.088624] Lustre: server umount lustre-OST0000 complete [ 3128.317217] LustreError: 93803:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3128.338508] LustreError: 93803:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 53 previous similar messages [ 3132.910140] LustreError: lustre-OST0001-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3132.924340] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 3132.934985] Lustre: Skipped 2 previous similar messages [ 3138.615280] Lustre: server umount lustre-OST0001 complete [ 3149.558796] Lustre: server umount lustre-MDT0000 complete [ 3153.582584] LustreError: 91604:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1769076905 with bad export cookie 5192416805444491206 [ 3153.596488] LustreError: MGC192.168.201.141@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3153.598752] LustreError: 91604:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 3154.373433] Lustre: server umount lustre-MDT0001 complete [ 3164.421164] Lustre: DEBUG MARKER: == conf-sanity test 85: osd_ost init: fail ea_fid_set ==== 05:15:14 (1769076914) [ 3174.355176] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3174.370156] Lustre: *** cfs_fail_loc=197, val=0*** [ 3190.048144] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 3195.169954] LustreError: 99250:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.201.141@tcp: failed processing log, type 4: rc = -110 [ 3200.337497] Lustre: *** cfs_fail_loc=197, val=0*** [ 3200.340525] Lustre: Skipped 65 previous similar messages [ 3220.898389] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3220.905342] Lustre: Skipped 3 previous similar messages [ 3228.582266] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 3235.418635] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3238.149667] Lustre: server umount lustre-OST0000 complete [ 3249.060756] Lustre: DEBUG MARKER: == conf-sanity test 86: Replacing mkfs.lustre -G option == 05:16:39 (1769076999) [ 3272.497820] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3279.523778] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 3289.187406] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 3294.301890] LustreError: 100792:0:(osp_object.c:617:osp_attr_get()) lustre-MDT0001-osp-MDT0000: osp_attr_get update error [0x200000009:0x1:0x0]: rc = -5 [ 3294.322408] LustreError: 100792:0:(lod_sub_object.c:917:lod_sub_prep_llog()) lustre-MDT0000-mdtlov: can't get id from catalogs: rc = -5 [ 3294.336639] LustreError: 100792:0:(lod_dev.c:510:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 21, retries 0, failed: rc = -5 [ 3294.863975] Lustre: server umount lustre-MDT0000 complete [ 3314.048973] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3335.968983] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_hostid [ 3348.093862] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing load_modules_local [ 3359.726045] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3366.603313] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3372.377809] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3380.679590] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3388.755741] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3388.847572] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3389.158159] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3389.221671] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3389.357584] Lustre: lustre-MDT0000: new disk, initializing [ 3389.526257] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3395.500124] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 3408.239221] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3408.358098] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3408.421171] Lustre: 104840:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 3408.707259] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3414.549568] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 3423.792503] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 3427.639912] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 3436.852213] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3436.934267] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3443.541456] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3445.039068] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 3454.523109] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3459.985622] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 3460.266505] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 3465.564309] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 3465.875822] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 3469.294411] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3469.304616] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3469.321971] Lustre: Skipped 4 previous similar messages [ 3469.331706] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3469.367690] Lustre: Skipped 7 previous similar messages [ 3474.177218] LustreError: 107779:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 3474.192474] LustreError: 107779:0:(obd_class.h:479:obd_check_dev()) Skipped 41 previous similar messages [ 3474.295098] Lustre: server umount lustre-OST0000 complete [ 3484.666550] LustreError: 104851:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3484.701078] LustreError: 104851:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 8 previous similar messages [ 3484.873641] Lustre: server umount lustre-MDT0000 complete [ 3489.478254] LustreError: 104830:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1769077241 with bad export cookie 5192416805444501839 [ 3489.482233] LustreError: MGC192.168.201.141@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3489.498374] LustreError: 104830:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 3489.974986] Lustre: server umount lustre-MDT0001 complete [ 3499.986128] Lustre: DEBUG MARKER: == conf-sanity test 87: check if MDT inode can hold EAs with N stripes properly ========================================================== 05:20:50 (1769077250) [ 3511.583782] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing unload_modules_local [ 3515.396608] Key type lgssc unregistered [ 3515.759402] LNet: 108910:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3515.772592] LNetError: 108910:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3515.795687] LNet: Removed LNI 192.168.201.141@tcp [ 3516.806537] Key type .llcrypt unregistered [ 3516.809885] Key type ._llcrypt unregistered [ 3537.487382] Key type ._llcrypt registered [ 3537.488838] Key type .llcrypt registered [ 3537.567926] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_hostid [ 3557.426805] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing load_modules_local [ 3558.910499] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3559.007215] alg: No test for adler32 (adler32-zlib) [ 3560.237158] Lustre: Lustre: Build Version: 2.17.50_3_g69d656b [ 3560.561166] LNet: Added LNI 192.168.201.141@tcp [8/256/0/180] [ 3562.273624] Key type lgssc registered [ 3563.434643] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3573.755477] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3580.787873] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3589.864045] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3599.045411] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3606.152713] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3615.038967] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3615.078730] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3615.086807] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3616.365604] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3616.402698] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3616.504413] Lustre: lustre-MDT0000: new disk, initializing [ 3616.595894] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3616.623499] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3621.837366] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 3634.232532] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3634.315914] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3634.762694] Lustre: lustre-OST0000: new disk, initializing [ 3634.770052] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3634.782553] Lustre: Skipped 1 previous similar message [ 3634.920987] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3642.813068] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 3644.547624] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 3644.566400] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 3644.592582] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 3654.535834] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3654.618285] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3654.834428] Lustre: lustre-OST0001: new disk, initializing [ 3654.840543] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 3654.991244] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 3662.276639] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 3663.970726] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:1:ost [ 3663.976438] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:1:ost] [ 3663.994418] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x280000400 [ 3674.084406] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3674.093760] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3674.110696] Lustre: Skipped 1 previous similar message [ 3674.134553] Lustre: Skipped 1 previous similar message [ 3675.514449] LustreError: 114499:0:(obd_class.h:479:obd_check_dev()) Device 17 not setup [ 3675.947860] Lustre: server umount lustre-MDT0000 complete [ 3685.273846] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: (null) [ 3691.048822] LustreError: 112254:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1769077443 with bad export cookie 12091467771037564319 [ 3691.057692] LustreError: MGC192.168.201.141@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3691.060394] LustreError: 112254:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 3691.122136] LustreError: 115095:0:(obd_class.h:479:obd_check_dev()) Device 12 not setup [ 3691.125315] LustreError: 115095:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 3691.158827] Lustre: server umount lustre-OST0000 complete [ 3694.892525] LustreError: 115295:0:(obd_class.h:479:obd_check_dev()) Device 2 not setup [ 3694.905610] LustreError: 115295:0:(obd_class.h:479:obd_check_dev()) Skipped 1 previous similar message [ 3695.108928] Lustre: server umount lustre-OST0001 complete [ 3707.044205] Lustre: DEBUG MARKER: == conf-sanity test 88: check the default mount options can be overridden ========================================================== 05:24:16 (1769077456) [ 3723.681249] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing load_modules_local [ 3735.624746] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3736.997890] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3742.984841] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3744.350220] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3753.278040] Lustre: DEBUG MARKER: == conf-sanity test 89: check tunefs --param and --erase-param{s} options ========================================================== 05:25:03 (1769077503) [ 3768.337491] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3770.749939] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3773.180323] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3775.433782] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3776.556899] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3779.145848] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3799.000486] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_hostid [ 3810.269155] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing load_modules_local [ 3820.956175] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3828.662950] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3835.362417] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3842.892307] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3850.321302] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3850.433825] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3850.697050] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3850.770336] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3850.918125] Lustre: lustre-MDT0000: new disk, initializing [ 3851.059401] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3851.090627] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3855.834052] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 3867.943916] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3867.989780] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3868.089714] Lustre: 120650:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 3868.158974] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 3868.164202] Lustre: Skipped 1 previous similar message [ 3868.234691] Lustre: lustre-MDT0001: new disk, initializing [ 3868.368617] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3868.449414] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 3868.461284] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3874.192902] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 3883.799825] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 3888.745976] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 3897.772639] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3897.869795] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3898.371288] Lustre: lustre-OST0000: new disk, initializing [ 3898.381650] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3898.680249] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3905.143545] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 3905.163312] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 3905.325936] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3907.475734] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 3916.532454] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount (FULL|IDLE) failover.node=192.0.2.254@tcp0,osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3921.883915] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 3922.149533] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 3928.065205] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 3928.466884] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 3936.224831] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3936.226472] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3936.231285] LustreError: Skipped 1 previous similar message [ 3936.263817] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3936.292592] Lustre: Skipped 1 previous similar message [ 3936.350050] Lustre: Skipped 1 previous similar message [ 3941.353479] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3941.377864] Lustre: Skipped 1 previous similar message [ 3945.951208] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 3946.085946] LustreError: 123592:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 3946.107802] LustreError: 123592:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 3946.705561] Lustre: server umount lustre-OST0000 complete [ 3955.689863] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3955.699224] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3955.708949] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3959.137921] LustreError: 123796:0:(obd_class.h:479:obd_check_dev()) Device 22 not setup [ 3959.157517] LustreError: 123796:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 3961.291746] Lustre: server umount lustre-MDT0000 complete [ 3961.830901] LustreError: 121645:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3961.857422] LustreError: 121645:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 4 previous similar messages [ 3965.608968] LustreError: 120642:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1769077717 with bad export cookie 12091467771037565544 [ 3965.610961] LustreError: MGC192.168.201.141@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3965.620830] LustreError: 120642:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 3966.001317] Lustre: server umount lustre-MDT0001 complete [ 3977.624571] Lustre: DEBUG MARKER: == conf-sanity test 90a: check max_mod_rpcs_in_flight is enforced ========================================================== 05:28:47 (1769077727) [ 3986.464677] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3987.089641] LustreError: 124670:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3987.205876] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3992.548304] LustreError: 124670:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3993.410945] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 3997.672726] LustreError: 124670:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4002.786364] LustreError: 124670:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4004.320446] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4010.208683] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 4017.805882] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 4022.930339] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 4031.800554] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4032.514643] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4032.523601] Lustre: Skipped 1 previous similar message [ 4040.262156] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 4051.103781] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4056.348457] Lustre: *** cfs_fail_loc=159, val=0*** [ 4073.010061] Lustre: lustre-MDT0000: Client 6e5272b5-6066-411b-81b7-c99692e99f09 (at 192.168.201.41@tcp) reconnecting [ 4075.037051] Lustre: *** cfs_fail_loc=159, val=0*** [ 4075.051943] Lustre: Skipped 5 previous similar messages [ 4091.368984] Lustre: lustre-MDT0000: Client 6e5272b5-6066-411b-81b7-c99692e99f09 (at 192.168.201.41@tcp) reconnecting [ 4095.968309] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4095.979032] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4095.988930] Lustre: Skipped 2 previous similar messages [ 4095.996648] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4096.006070] Lustre: Skipped 3 previous similar messages [ 4097.862449] LustreError: 127162:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 4097.876204] LustreError: 127162:0:(obd_class.h:479:obd_check_dev()) Skipped 13 previous similar messages [ 4098.263602] Lustre: server umount lustre-OST0000 complete [ 4106.212845] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4106.216777] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4106.236404] Lustre: Skipped 1 previous similar message [ 4106.262963] Lustre: Skipped 1 previous similar message [ 4108.935802] Lustre: server umount lustre-MDT0000 complete [ 4111.342131] LustreError: 125833:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4111.365961] LustreError: 125833:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 4113.138525] LustreError: 124649:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1769077865 with bad export cookie 12091467771037566419 [ 4113.170516] LustreError: MGC192.168.201.141@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4113.519186] Lustre: server umount lustre-MDT0001 complete [ 4125.481639] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing unload_modules_local [ 4129.359216] Key type lgssc unregistered [ 4129.688345] LNet: 128098:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4129.703649] LNetError: 128098:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4129.719841] LNet: Removed LNI 192.168.201.141@tcp [ 4130.801966] Key type .llcrypt unregistered [ 4130.805740] Key type ._llcrypt unregistered [ 4159.566463] Key type ._llcrypt registered [ 4159.571795] Key type .llcrypt registered [ 4159.707737] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing load_modules_local [ 4161.430749] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4161.475432] alg: No test for adler32 (adler32-zlib) [ 4162.633117] Lustre: Lustre: Build Version: 2.17.50_3_g69d656b [ 4162.987968] LNet: Added LNI 192.168.201.141@tcp [8/256/0/180] [ 4164.632070] Key type lgssc registered [ 4166.205141] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4174.771423] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4174.784728] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4176.686427] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4182.874094] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 4186.096028] LustreError: 129498:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4186.121618] LustreError: 129498:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 4191.208416] LustreError: 129498:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4194.626162] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4195.071711] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4199.491689] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 4205.635809] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 4210.836959] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 4218.360090] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4218.875453] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4220.920487] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:42 to 0x280000401:65) [ 4226.917464] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 4233.571055] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4244.126919] Lustre: *** cfs_fail_loc=159, val=0*** [ 4260.336681] Lustre: lustre-MDT0000: Client ed902ead-9ed7-4b4f-94d2-deca046177c2 (at 192.168.201.41@tcp) reconnecting [ 4263.358174] Lustre: *** cfs_fail_loc=159, val=0*** [ 4279.830109] Lustre: lustre-MDT0001: Client ed902ead-9ed7-4b4f-94d2-deca046177c2 (at 192.168.201.41@tcp) reconnecting [ 4281.185219] Lustre: *** cfs_fail_loc=159, val=0*** [ 4281.190292] Lustre: Skipped 3 previous similar messages [ 4297.717820] Lustre: lustre-MDT0001: Client ed902ead-9ed7-4b4f-94d2-deca046177c2 (at 192.168.201.41@tcp) reconnecting [ 4303.920641] Lustre: 132205:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4308.360638] Lustre: *** cfs_fail_loc=159, val=0*** [ 4308.365922] Lustre: Skipped 4 previous similar messages [ 4324.957244] Lustre: lustre-MDT0000: Client c4e11ba4-bb87-47a5-a6ee-844ac6f3617e (at 192.168.201.41@tcp) reconnecting [ 4326.998899] Lustre: *** cfs_fail_loc=159, val=0*** [ 4327.000625] Lustre: Skipped 14 previous similar messages [ 4343.288866] Lustre: lustre-MDT0000: Client c4e11ba4-bb87-47a5-a6ee-844ac6f3617e (at 192.168.201.41@tcp) reconnecting [ 4344.857305] Lustre: 132500:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4354.019518] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4354.028369] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4354.035372] LustreError: Skipped 1 previous similar message [ 4354.042387] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4354.053185] Lustre: Skipped 1 previous similar message [ 4357.320257] LustreError: 132598:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 4357.456798] Lustre: server umount lustre-OST0000 complete [ 4364.237231] LustreError: 132800:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup [ 4364.255039] LustreError: 132800:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 4364.266503] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4364.296937] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4364.328503] LustreError: 130507:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4364.363742] LustreError: 130507:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 4364.592045] Lustre: server umount lustre-MDT0000 complete [ 4368.883742] LustreError: 129477:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1769078120 with bad export cookie 17094002310560045609 [ 4368.891030] LustreError: MGC192.168.201.141@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4368.911191] LustreError: 129477:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 4369.176573] LustreError: 133003:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [ 4369.187965] LustreError: 133003:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 4369.718725] Lustre: server umount lustre-MDT0001 complete [ 4383.933650] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing unload_modules_local [ 4387.778486] Key type lgssc unregistered [ 4388.543502] LNet: 133536:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4388.549019] LNetError: 133536:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4389.611518] LNet: Removed LNI 192.168.201.141@tcp [ 4390.726453] Key type .llcrypt unregistered [ 4390.731329] Key type ._llcrypt unregistered [ 4418.429397] Key type ._llcrypt registered [ 4418.433840] Key type .llcrypt registered [ 4418.564747] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing load_modules_local [ 4420.054894] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4420.188321] alg: No test for adler32 (adler32-zlib) [ 4421.586175] Lustre: Lustre: Build Version: 2.17.50_3_g69d656b [ 4421.942360] LNet: Added LNI 192.168.201.141@tcp [8/256/0/180] [ 4423.663286] Key type lgssc registered [ 4424.598501] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4433.717585] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4433.753262] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4435.440243] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4440.860445] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 4445.666660] LustreError: 134950:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4445.712504] LustreError: 134950:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 4446.714222] LustreError: 134950:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4451.421941] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4451.745551] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4455.762641] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 4461.199352] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 4464.933044] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 4472.351946] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4472.599188] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4476.719997] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:97) [ 4479.054982] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 4481.850103] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:129) [ 4486.920576] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4495.728013] Lustre: 137202:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4500.923173] Lustre: 137251:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4503.339386] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 in log params [ 4504.566368] Lustre: Setting parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=8 in log params [ 4509.273058] Lustre: 137399:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre/mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 (mode = 0) failed: rc = -17 [ 4510.259617] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=7 in log params [ 4511.509958] Lustre: 137497:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4517.856190] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4517.868656] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4517.875275] LustreError: Skipped 1 previous similar message [ 4517.882393] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4517.892531] Lustre: Skipped 1 previous similar message [ 4517.918091] Lustre: Skipped 1 previous similar message [ 4522.115582] LustreError: 137605:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 4522.287431] Lustre: server umount lustre-OST0000 complete [ 4528.104150] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4528.108975] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4528.119252] Lustre: Skipped 1 previous similar message [ 4533.037706] LustreError: 137806:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup [ 4533.049326] LustreError: 137806:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 4533.222610] LustreError: 135663:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4533.293716] Lustre: server umount lustre-MDT0000 complete [ 4537.742110] LustreError: 137361:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1769078289 with bad export cookie 2850723155223020447 [ 4537.743747] LustreError: MGC192.168.201.141@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4537.752243] LustreError: 137361:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 4537.925342] LustreError: 138008:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [ 4537.947959] LustreError: 138008:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 4538.245338] Lustre: server umount lustre-MDT0001 complete [ 4552.600351] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing unload_modules_local [ 4556.658834] Key type lgssc unregistered [ 4557.086798] LNet: 138540:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4557.098556] LNetError: 138540:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4557.122882] LNet: Removed LNI 192.168.201.141@tcp [ 4558.243722] Key type .llcrypt unregistered [ 4558.245899] Key type ._llcrypt unregistered [ 4589.453708] Key type ._llcrypt registered [ 4589.461603] Key type .llcrypt registered [ 4589.719626] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing load_modules_local [ 4591.433058] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4591.617748] alg: No test for adler32 (adler32-zlib) [ 4593.010424] Lustre: Lustre: Build Version: 2.17.50_3_g69d656b [ 4593.351901] LNet: Added LNI 192.168.201.141@tcp [8/256/0/180] [ 4595.087167] Key type lgssc registered [ 4596.624297] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4605.652599] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4605.675847] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4607.449872] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4612.827160] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 4617.697761] LustreError: 139939:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4617.734867] LustreError: 139939:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 4622.819719] LustreError: 139939:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4623.531305] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4624.222044] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4629.482910] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 4638.098829] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 4642.631536] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 4651.437326] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4651.948894] Lustre: lustre-OST0000: Not available for connect from 0@lo (not set up) [ 4652.143548] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4657.156033] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:161) [ 4659.389573] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 4666.140962] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4669.463831] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:129) [ 4675.063309] Lustre: *** cfs_fail_loc=159, val=0*** [ 4691.520741] Lustre: lustre-MDT0000: Client 12bccf48-e855-412d-b222-0173323f6db2 (at 192.168.201.41@tcp) reconnecting [ 4698.086423] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4698.097302] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4698.125022] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4700.129200] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4700.133464] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4700.152810] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4703.752192] LustreError: 142320:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 4704.028322] Lustre: server umount lustre-OST0000 complete [ 4710.376991] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4710.399776] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4710.405893] Lustre: Skipped 1 previous similar message [ 4710.436633] Lustre: Skipped 2 previous similar messages [ 4714.349284] LustreError: 142523:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup [ 4714.359688] LustreError: 142523:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 4714.438905] Lustre: server umount lustre-MDT0000 complete [ 4715.494272] LustreError: 139940:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4715.525536] LustreError: 139940:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 4718.485575] LustreError: 141570:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1769078470 with bad export cookie 11884272298984881814 [ 4718.489629] LustreError: MGC192.168.201.141@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4718.495419] LustreError: 141570:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 4718.662953] LustreError: 142724:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [ 4718.672519] LustreError: 142724:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 4718.829757] Lustre: server umount lustre-MDT0001 complete [ 4727.512500] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing unload_modules_local [ 4730.833168] Key type lgssc unregistered [ 4731.104352] LNet: 143258:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4731.117497] LNetError: 143258:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4731.139461] LNet: Removed LNI 192.168.201.141@tcp [ 4732.202950] Key type .llcrypt unregistered [ 4732.204560] Key type ._llcrypt unregistered [ 4757.694055] Key type ._llcrypt registered [ 4757.696310] Key type .llcrypt registered [ 4757.800453] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing load_modules_local [ 4758.593842] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4758.763461] alg: No test for adler32 (adler32-zlib) [ 4759.828523] Lustre: Lustre: Build Version: 2.17.50_3_g69d656b [ 4760.050444] LNet: Added LNI 192.168.201.141@tcp [8/256/0/180] [ 4761.703178] Key type lgssc registered [ 4762.890588] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4770.708639] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4770.729576] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4772.365898] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4776.761387] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 4782.581445] LustreError: 144652:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4782.621065] LustreError: 144652:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 4787.427260] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4787.726843] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4794.842875] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 4801.433399] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 4805.354390] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 4815.295177] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4815.790107] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4817.833975] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:170 to 0x280000401:193) [ 4821.181388] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 4826.711802] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4832.175956] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:161) [ 4851.020969] Lustre: lustre-OST0000: evicting 7d171e34-27d6-4b59-9f3a-969f60e01b2e (at 192.168.201.41@tcp) by administrative request [ 4851.030034] Lustre: Skipped 1 previous similar message [ 4883.427488] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4883.437504] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4883.446108] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4883.456424] Lustre: Skipped 1 previous similar message [ 4886.332156] LustreError: 147439:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 4886.431831] Lustre: server umount lustre-OST0000 complete [ 4893.669774] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4893.681048] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4893.681783] Lustre: Skipped 1 previous similar message [ 4893.690232] Lustre: Skipped 1 previous similar message [ 4897.079338] LustreError: 147640:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup [ 4897.089388] LustreError: 147640:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 4897.304260] Lustre: server umount lustre-MDT0000 complete [ 4898.793485] LustreError: 145370:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4898.832070] LustreError: 145370:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 4901.037782] LustreError: 144632:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1769078653 with bad export cookie 4214434224161456193 [ 4901.046136] LustreError: MGC192.168.201.141@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4901.255379] LustreError: 147842:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [ 4901.264780] LustreError: 147842:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 4901.487380] Lustre: server umount lustre-MDT0001 complete [ 4913.375077] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing unload_modules_local [ 4916.046630] Key type lgssc unregistered [ 4916.328953] LNet: 148375:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4916.347443] LNetError: 148375:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4916.365960] LNet: Removed LNI 192.168.201.141@tcp [ 4917.366900] Key type .llcrypt unregistered [ 4917.368424] Key type ._llcrypt unregistered [ 4996.491307] Key type ._llcrypt registered [ 4996.494239] Key type .llcrypt registered [ 4996.608276] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing load_modules_local [ 4997.832743] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4997.955349] alg: No test for adler32 (adler32-zlib) [ 4999.028978] Lustre: Lustre: Build Version: 2.17.50_3_g69d656b [ 4999.315966] LNet: Added LNI 192.168.201.141@tcp [8/256/0/180] [ 5001.047224] Key type lgssc registered [ 5002.515229] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5011.570616] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5011.596305] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5013.350760] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5017.250531] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 5023.717063] LustreError: 150853:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5023.751341] LustreError: 150853:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 5027.337191] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5028.392079] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5035.762175] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 5042.569147] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 5047.566773] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 5054.358938] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5054.608163] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5058.748986] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:195 to 0x280000401:225) [ 5061.374937] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 5067.302764] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 5073.096240] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:193) [ 5083.617605] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5083.619087] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5083.630787] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 5083.638674] Lustre: Skipped 1 previous similar message [ 5083.667908] Lustre: Skipped 1 previous similar message [ 5087.481290] LustreError: 153135:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 5087.571820] Lustre: server umount lustre-OST0000 complete [ 5093.871165] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5093.880550] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5093.881593] Lustre: Skipped 1 previous similar message [ 5093.889216] Lustre: Skipped 1 previous similar message [ 5098.981228] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5098.985480] Lustre: Skipped 1 previous similar message [ 5103.075691] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5106.143674] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 5106.294172] LustreError: 153337:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup [ 5106.301493] LustreError: 153337:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 5106.396803] Lustre: server umount lustre-MDT0000 complete [ 5108.195313] LustreError: 150848:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5108.222574] LustreError: 150848:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 5111.699622] LustreError: 150834:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1769078863 with bad export cookie 7249434393630674270 [ 5111.704543] LustreError: MGC192.168.201.141@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5111.707708] LustreError: 150834:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 5111.937757] LustreError: 153540:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [ 5111.965503] LustreError: 153540:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 5112.802665] Lustre: server umount lustre-MDT0001 complete [ 5129.707911] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing unload_modules_local [ 5133.254161] Key type lgssc unregistered [ 5133.603742] LNet: 154073:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5133.613276] LNetError: 154073:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5133.644451] LNet: Removed LNI 192.168.201.141@tcp [ 5134.652687] Key type .llcrypt unregistered [ 5134.654991] Key type ._llcrypt unregistered [ 5165.778061] Key type ._llcrypt registered [ 5165.779543] Key type .llcrypt registered [ 5165.986598] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing load_modules_local [ 5167.445702] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5167.600606] alg: No test for adler32 (adler32-zlib) [ 5168.889853] Lustre: Lustre: Build Version: 2.17.50_3_g69d656b [ 5169.221423] LNet: Added LNI 192.168.201.141@tcp [8/256/0/180] [ 5170.967222] Key type lgssc registered [ 5172.296805] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5181.449211] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5191.698788] Lustre: DEBUG MARKER: == conf-sanity test 100: check lshowmount lists MGS, MDT, OST and 0@lo ========================================================== 05:49:01 (1769078941) [ 5214.843231] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_hostid [ 5228.729724] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing load_modules_local [ 5239.696294] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5248.333616] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5256.061467] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5263.512654] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5273.108285] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5273.195570] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5273.217549] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5274.692295] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 5274.832516] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 5275.052851] Lustre: lustre-MDT0000: new disk, initializing [ 5275.213407] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5275.256242] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 5279.824291] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 5291.903835] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5291.945447] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5292.024387] Lustre: 158307:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 5292.080997] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 5292.084796] Lustre: Skipped 1 previous similar message [ 5292.293127] Lustre: lustre-MDT0001: new disk, initializing [ 5292.423994] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5292.475536] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 5292.486804] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 5297.633704] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 5306.762346] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 5311.101523] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 5318.778033] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5318.913886] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5319.270454] Lustre: lustre-OST0000: new disk, initializing [ 5319.273721] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 5319.379974] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5324.924980] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 5324.938666] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 5325.065951] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 5327.144741] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 5335.602565] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 5350.881954] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5350.895079] Lustre: Skipped 1 previous similar message [ 5350.896767] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5350.909942] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 5353.992165] LustreError: 160894:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 5354.152816] Lustre: server umount lustre-OST0000 complete [ 5359.072764] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 5359.073127] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5359.124740] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5359.138256] Lustre: Skipped 2 previous similar messages [ 5361.120874] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5361.136710] Lustre: Skipped 1 previous similar message [ 5364.343524] LustreError: 161095:0:(obd_class.h:479:obd_check_dev()) Device 22 not setup [ 5364.346381] LustreError: 161095:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 5364.472128] Lustre: server umount lustre-MDT0000 complete [ 5369.922865] LustreError: 158296:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1769079122 with bad export cookie 199114103287102376 [ 5369.934707] LustreError: 158296:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 1 previous similar message [ 5369.940229] LustreError: MGC192.168.201.141@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5370.063545] LustreError: 161298:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 5370.068967] LustreError: 161298:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 5370.364582] Lustre: server umount lustre-MDT0001 complete [ 5386.179542] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing unload_modules_local [ 5392.890434] Key type lgssc unregistered [ 5393.744492] LNet: 161833:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5393.760582] LNetError: 161833:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5393.805454] LNet: Removed LNI 192.168.201.141@tcp [ 5396.068569] Key type .llcrypt unregistered [ 5396.075465] Key type ._llcrypt unregistered [ 5426.237805] Key type ._llcrypt registered [ 5426.239457] Key type .llcrypt registered [ 5426.359945] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing load_modules_local [ 5427.512490] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5427.549990] alg: No test for adler32 (adler32-zlib) [ 5428.806534] Lustre: Lustre: Build Version: 2.17.50_3_g69d656b [ 5429.087842] LNet: Added LNI 192.168.201.141@tcp [8/256/0/180] [ 5430.799117] Key type lgssc registered [ 5431.920974] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5439.824876] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5439.851849] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5441.332729] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5446.104691] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 5451.758406] LustreError: 163232:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5451.785466] LustreError: 163232:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 5455.539975] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5455.989936] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5460.491216] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 5466.143337] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 5471.278856] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 5479.966925] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5480.524767] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5486.707422] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:3 to 0x280000401:33) [ 5488.174378] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 5495.467713] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 5502.971221] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5503.007525] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5503.031750] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5503.043605] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5503.062191] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5504.296968] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5504.336199] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5504.376229] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5504.387741] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5504.406764] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5505.844781] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5505.864272] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5505.876885] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5505.884733] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5505.905877] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5506.764486] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5507.803329] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5508.932781] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5508.946390] Lustre: Skipped 2 previous similar messages [ 5508.959936] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5508.974278] Lustre: Skipped 2 previous similar messages [ 5508.984795] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5508.992996] Lustre: Skipped 2 previous similar messages [ 5508.998086] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5509.010356] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5509.013825] Lustre: Skipped 2 previous similar messages [ 5510.120527] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5511.344800] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5512.342374] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5513.414939] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5513.428309] Lustre: Skipped 3 previous similar messages [ 5513.450978] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5513.458725] Lustre: Skipped 3 previous similar messages [ 5513.460840] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5513.465633] Lustre: Skipped 3 previous similar messages [ 5513.467878] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5513.480636] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5513.487913] Lustre: Skipped 3 previous similar messages [ 5514.752933] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5515.867823] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5516.755845] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5517.467298] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5518.341361] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5519.383839] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5520.416554] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5521.444547] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5521.449695] Lustre: Skipped 7 previous similar messages [ 5521.469976] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5521.477738] Lustre: Skipped 7 previous similar messages [ 5521.481686] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5521.486597] Lustre: Skipped 7 previous similar messages [ 5521.490755] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5521.504643] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5521.510996] Lustre: Skipped 7 previous similar messages [ 5522.562299] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5523.549798] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5524.668647] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5525.828242] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5527.162868] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5528.117457] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5529.212296] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5530.124676] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5531.044172] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5532.440882] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5533.538358] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5534.575689] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5535.687865] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5536.713417] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5538.189578] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5538.198381] Lustre: Skipped 14 previous similar messages [ 5538.217066] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5538.230960] Lustre: Skipped 14 previous similar messages [ 5538.241236] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5538.251548] Lustre: Skipped 14 previous similar messages [ 5538.261672] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5538.279571] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5538.283751] Lustre: Skipped 14 previous similar messages [ 5539.763028] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5540.880748] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5542.011951] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5543.021736] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5543.759151] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5544.665438] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5546.251537] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5547.477112] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5548.445134] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5549.662731] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5550.925789] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5552.334890] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5553.351664] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5554.407661] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5555.499091] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5556.767855] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5557.776413] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5558.712070] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5559.729662] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5560.820430] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5561.822345] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5563.011941] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5564.012233] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5564.969382] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5566.014882] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5567.031191] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5568.015149] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5568.909931] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5569.985101] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5570.865218] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5570.868199] Lustre: Skipped 29 previous similar messages [ 5570.878568] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5570.891506] Lustre: Skipped 29 previous similar messages [ 5570.900431] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5570.908956] Lustre: Skipped 29 previous similar messages [ 5570.914124] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5570.928335] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5570.941514] Lustre: Skipped 29 previous similar messages [ 5572.011035] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5572.902959] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5573.936553] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5575.202707] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5576.314995] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5577.392795] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5578.298335] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5579.383546] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5580.363743] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5581.463231] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5582.517668] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5583.572441] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5584.549178] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5585.456958] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5586.601231] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5587.655168] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5588.849698] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5590.148642] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5591.134000] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5592.259747] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5593.138475] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5594.090965] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5594.996542] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5595.954432] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5596.913535] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5598.023094] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5599.089876] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5600.055716] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5601.149674] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5602.070523] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5603.071831] LustreError: 163252:0:(osp_precreate.c:651:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5603.085409] LustreError: 163252:0:(osp_precreate.c:1355:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5603.102299] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5604.046742] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5605.094481] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5605.902365] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5606.796574] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5607.684658] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5608.809820] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5609.758805] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5610.770897] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5611.941122] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5612.998272] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5614.054806] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5615.022337] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5616.302945] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5617.509642] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5618.411381] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5619.354838] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5620.389210] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5621.650540] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5622.613802] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5623.756694] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5624.880155] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5626.133971] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5627.145185] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5628.158347] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5629.269821] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5630.286235] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5631.434775] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5632.469476] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5633.586569] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5634.438334] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5635.364314] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5635.372622] Lustre: Skipped 61 previous similar messages [ 5635.385982] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5635.406956] Lustre: Skipped 61 previous similar messages [ 5635.411510] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5635.418908] Lustre: Skipped 61 previous similar messages [ 5635.422813] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5635.433568] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5635.439237] Lustre: Skipped 61 previous similar messages [ 5636.332903] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5637.207313] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5637.990610] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5638.899538] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5639.839209] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5640.737753] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5641.581905] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5642.576180] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5643.329816] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5644.141919] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5645.009321] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5645.997321] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5646.912226] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5647.892879] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5648.845772] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5649.824500] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5650.591411] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5651.654811] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5652.647048] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5653.517064] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5654.367576] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5655.063276] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5656.003169] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5656.954802] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5657.774662] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5658.840561] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5659.784417] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5660.832049] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5661.688196] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5662.628605] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5663.657580] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5664.643913] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5665.628244] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5666.875752] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5668.024684] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5669.074237] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5670.232953] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5671.209029] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5672.003283] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5673.005096] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5673.957856] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5674.789971] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5675.491036] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5676.374070] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5677.477463] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5678.477254] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5679.466578] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5680.492702] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5681.509816] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5682.479194] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5683.429905] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5684.488319] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5685.472785] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5686.309309] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5687.281814] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5688.224570] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5689.313851] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5690.228391] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5691.121692] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5692.124474] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5693.135925] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5694.054296] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5694.908281] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5695.764079] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5696.740235] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5697.735521] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5698.600643] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5699.440794] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5700.304785] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5701.035852] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5701.861978] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5702.794644] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5703.697302] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5704.622874] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5705.404799] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5706.307120] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5707.044819] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5707.801965] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5708.643707] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5709.479690] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5710.355723] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5711.185146] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5712.092096] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5712.960439] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5713.925326] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5714.751196] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5715.678881] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5716.590491] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5717.592817] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5718.571671] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5719.530886] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5720.362504] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5721.228279] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5722.124175] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5722.868627] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5723.728668] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5724.731557] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5725.648319] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5726.424753] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5727.337824] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5728.131144] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5729.155959] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5730.179693] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5731.095936] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5731.956491] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5733.151280] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5734.075106] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5735.061855] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5735.918207] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5736.734089] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5737.590321] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5738.450100] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5739.243752] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5740.064160] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5740.940218] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5741.829356] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5742.698959] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5743.490224] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5744.265531] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5745.074418] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5745.926032] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5746.608528] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5747.476330] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5748.226646] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5749.080383] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5749.990740] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5750.914735] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5751.855454] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5752.778663] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5753.588879] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5754.485291] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5755.217255] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5756.025626] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5756.848139] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5757.611058] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5758.385480] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5759.329568] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5760.228838] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5761.017926] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5761.780509] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5762.585601] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5763.248952] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5763.947330] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5763.950543] Lustre: Skipped 142 previous similar messages [ 5763.962862] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5763.972365] Lustre: Skipped 142 previous similar messages [ 5763.975374] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5763.980899] Lustre: Skipped 142 previous similar messages [ 5763.984352] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5763.992430] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5764.004137] Lustre: Skipped 142 previous similar messages [ 5764.638182] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5765.552935] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5766.373476] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5767.194412] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5767.877704] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5768.693747] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5769.603169] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5770.528176] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5771.280142] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5772.118210] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5772.940057] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5773.906134] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5774.818488] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5775.646616] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5776.420590] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5777.223423] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5777.924975] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5778.670215] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5779.486518] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5780.404664] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5781.350746] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5782.250534] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5783.227346] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5783.949509] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5784.862379] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5785.897738] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5786.873793] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5787.860438] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5788.865037] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5789.788157] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5790.732688] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5791.613640] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5792.584989] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5793.614684] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5794.331843] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5795.166649] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5796.113261] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5796.874463] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5797.840922] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5798.663396] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5799.652557] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5800.462631] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5801.503851] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5802.223794] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5803.110661] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5803.905169] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5804.812970] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5805.657853] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5806.557220] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5807.385707] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5808.274202] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5809.115257] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5810.036219] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5810.759530] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5811.706936] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5812.408350] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5813.215610] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5814.053891] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5814.860189] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5815.512575] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5816.247429] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5816.984665] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5817.719334] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5818.420118] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5819.124743] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5820.028420] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5820.893797] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5821.736074] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5822.678439] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5823.399552] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5824.123175] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5824.839828] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5825.704479] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5826.416308] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5827.183980] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5828.005573] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5828.747669] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5829.505140] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5830.423929] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5831.203452] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5832.123652] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5832.838169] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5833.696589] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5834.478067] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5835.331895] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5836.078421] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5836.778811] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5837.801442] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5838.592360] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5839.439132] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5840.214227] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5841.050593] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5841.889490] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5842.633314] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5843.332178] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5844.227709] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5844.959631] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5845.916052] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5846.875448] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5847.783972] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5848.556755] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5849.461041] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5850.300657] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5851.210429] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5852.021155] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5853.036778] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5854.028845] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5854.870479] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5855.689450] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5856.584521] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5857.328193] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5858.235908] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5859.132059] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5859.992213] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5860.791350] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5861.516550] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5862.339988] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5863.174112] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5863.969183] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5864.945830] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5865.827492] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5866.889911] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5867.690460] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5868.620669] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5869.267605] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5870.088844] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5870.852740] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5871.690572] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5872.460634] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5873.322226] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5874.131435] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5874.987863] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5875.953045] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5876.924657] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5877.796979] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5878.783588] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5879.604535] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5880.446835] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5881.277666] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5882.148103] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5883.141082] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5883.932019] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5884.808296] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5885.843390] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5886.739821] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5887.661365] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5888.546638] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5889.433446] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5890.222656] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5890.980850] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5891.824547] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5892.721724] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5893.651270] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5894.530272] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5895.513720] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5896.432811] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5897.363743] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5898.235900] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5899.108940] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5899.966632] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5900.880075] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5901.856759] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5902.894539] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5903.775748] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5904.626345] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5905.600943] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5906.560187] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5907.405695] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5908.438212] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5909.475439] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5910.567291] LustreError: 163252:0:(osp_precreate.c:651:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5910.581868] LustreError: 163252:0:(osp_precreate.c:1355:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5910.585540] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5911.408341] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5912.480455] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5913.364482] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5914.349636] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5915.156840] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5916.102297] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5916.973665] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5917.941917] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5918.742994] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5919.676921] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5920.339933] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5921.171227] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5921.877151] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5922.780306] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5923.654510] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5924.405352] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5925.133861] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5925.840409] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5926.624107] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5927.409139] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5928.157185] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5929.007266] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5929.973069] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5930.856081] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5931.703347] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5932.584828] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5933.307030] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5934.077422] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5934.789549] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5935.779519] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5936.555485] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5937.300874] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5938.039575] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5938.952208] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5939.800467] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5940.591453] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5941.409653] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5942.228651] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5942.956252] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5943.790816] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5944.528765] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5945.330277] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5946.085414] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5946.867346] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5947.689890] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5948.588422] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5949.412047] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5950.261930] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5951.106747] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5952.119089] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5952.936873] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5953.819359] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5954.739652] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5955.790079] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5956.552687] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5957.410114] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5958.230407] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5959.085876] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5959.975303] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5961.002612] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5961.942243] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5962.905068] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5963.708758] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5964.461968] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5965.264473] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5966.096549] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5966.901421] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5967.851659] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5968.633993] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5969.455970] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5970.223144] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5971.064749] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5971.824905] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5972.690575] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5973.519670] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5974.478821] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5975.317334] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5976.470674] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5977.390583] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5978.334050] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5979.128220] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5979.921488] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5980.751601] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5981.597299] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5982.365985] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5983.423535] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5984.284484] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5985.206520] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5986.261941] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5987.118708] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5988.150929] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5989.070920] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5989.792655] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5990.654068] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5991.589191] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5992.407764] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5993.030723] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5993.940155] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5994.898801] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5995.796070] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5996.723936] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5997.669208] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5998.431058] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5999.138467] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5999.959250] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6000.887704] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6001.677234] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6002.507430] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6003.315933] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6004.175148] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6005.030566] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6005.884383] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6006.657697] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6007.422232] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6008.118101] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6008.937912] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6009.759755] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6010.552888] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6011.275633] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6012.043781] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6012.925044] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6013.813153] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6014.639828] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6015.452877] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6016.615870] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6017.544149] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6018.896463] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6019.976173] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 6019.979728] Lustre: Skipped 298 previous similar messages [ 6020.000936] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6020.016555] Lustre: Skipped 298 previous similar messages [ 6020.019983] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 6020.025458] Lustre: Skipped 298 previous similar messages [ 6020.028424] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6020.035692] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 6020.039642] Lustre: Skipped 298 previous similar messages [ 6021.155780] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6022.235855] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6023.106524] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6023.987173] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6025.055643] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6025.816614] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6026.648816] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6027.493426] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6028.382182] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6029.115588] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6030.043719] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6031.067954] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6031.949146] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6032.770796] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6033.783244] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6034.619571] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6035.544383] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6036.489041] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6037.460151] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6038.411911] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6039.328640] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6040.598348] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6041.733219] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6042.943578] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6044.059988] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6044.961194] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6045.902348] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6046.701270] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6047.492751] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6048.301145] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6049.146164] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6050.121342] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6051.047190] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6051.969340] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6052.929796] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6053.883671] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6054.704291] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6055.736312] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6056.472451] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6057.357592] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6058.240739] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6059.103480] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6059.843807] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6060.699518] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6061.538691] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6062.324990] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6063.216073] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6064.088325] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6064.989831] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6065.931037] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6066.702621] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6067.534082] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6068.255319] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6069.176082] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6070.128915] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6071.147213] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6072.088713] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6073.036390] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6074.015515] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6075.015314] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6075.955076] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6076.870612] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6077.819742] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6078.902648] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6079.915608] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6080.838548] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6081.509275] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6082.447295] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6083.347736] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6084.290422] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6085.289447] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6086.096300] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6087.075585] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6088.074598] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6088.975718] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6089.980801] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6090.975642] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6091.889418] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6092.913545] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6093.666669] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6094.679344] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6095.683734] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6096.655707] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6097.661531] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6098.591461] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6099.512054] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6100.503736] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6101.306809] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6102.097703] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6102.811734] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6103.617670] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6104.305713] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6105.142383] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6106.017602] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6106.738421] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6107.575624] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6108.324354] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6109.191209] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6109.967983] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6110.756707] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6111.441745] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6112.235843] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6112.974981] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6113.983856] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6114.863823] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6115.619396] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6116.457729] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6117.329625] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6117.979310] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6118.870084] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6119.682251] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6120.671077] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6121.356729] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6122.115885] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6122.975322] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6123.809982] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6124.688402] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6125.423985] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6126.201664] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6127.057557] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6127.901856] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6128.642396] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6129.435347] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6130.282190] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6131.043702] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6131.805020] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6132.566771] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6133.427596] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6134.270452] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6135.039176] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6135.765195] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6136.655195] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6137.545360] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6138.439396] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6139.256213] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6140.094505] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6141.111848] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6141.866925] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6142.781952] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6143.656690] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6144.414442] LustreError: 163252:0:(osp_precreate.c:651:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 6144.426309] LustreError: 163252:0:(osp_precreate.c:1355:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 6144.439762] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6145.203710] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6146.031490] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6146.841163] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6147.679206] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6148.433172] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6149.392975] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6150.239320] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6151.035445] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6151.863889] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6152.594426] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6153.306795] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6154.019852] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6154.759408] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6155.591492] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6156.365628] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6157.152832] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6157.901050] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6158.595209] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6159.637331] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6160.356142] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6161.147260] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6161.825355] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6162.502271] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6163.325435] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6164.249605] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6165.612717] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6166.409701] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6167.250267] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6168.186132] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6169.052541] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6169.867429] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6170.660699] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6171.596280] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6172.375966] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6173.172084] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6173.764101] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6174.468255] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6175.142758] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6175.860376] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6176.628672] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6177.387827] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6178.108750] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6179.014291] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6179.695467] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6180.424023] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6181.012267] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6181.978791] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6182.849745] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6183.927762] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6184.971563] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6185.778571] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6186.617726] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6187.554263] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6188.460898] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6189.477837] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6190.354977] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6191.203450] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6192.006376] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6192.839543] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6193.801757] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6194.768320] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6195.578849] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6196.555766] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6197.358661] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6198.305910] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6199.214576] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6200.108629] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6200.872269] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6201.741948] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6202.578620] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6203.358307] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6204.235334] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6205.013511] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6205.667646] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6206.530800] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6207.486397] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6208.305374] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6209.043305] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6210.003094] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6210.837552] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6211.651364] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6212.527057] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6213.483079] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6214.242032] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6215.067768] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6215.893386] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6216.698427] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6217.560297] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6218.251261] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6218.943339] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6219.821473] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6220.532659] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6221.368913] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6222.128945] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6222.971994] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6223.755152] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6224.540688] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6225.239901] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6226.082557] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6226.775384] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6227.544218] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6228.320951] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6229.306482] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6230.137840] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6230.808411] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6231.680256] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6232.508844] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6233.148210] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6233.858723] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6234.630709] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6235.351472] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6236.163704] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6236.874626] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6237.695569] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6238.461354] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6239.225062] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6240.080388] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6240.951443] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6241.811926] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6242.650323] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6243.529420] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6244.444240] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6245.153439] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6245.942978] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6246.763575] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6247.500676] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6248.251949] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6248.896182] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6249.834346] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6250.515880] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6251.271473] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6252.120083] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6252.901239] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6253.407627] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6254.167488] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6254.964987] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6255.731051] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6256.574325] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6257.658070] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6258.414143] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6259.297961] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6260.079100] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6260.990421] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6261.749800] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6262.556249] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6263.320207] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6264.261831] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6265.019484] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6265.927696] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6266.658312] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6267.597154] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6268.330763] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6269.336589] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6270.050614] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6270.810877] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6271.539541] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6272.472542] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6273.318261] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6274.205685] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6275.050759] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6275.922474] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6276.723096] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6277.516959] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6278.308987] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6279.149328] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6279.893203] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6280.674920] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6281.342962] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6282.091287] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6282.819714] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6283.503489] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6284.342473] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6285.200223] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6285.917335] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6286.701414] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6287.446899] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6288.270623] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6288.998996] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6289.758830] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6290.622146] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6291.377797] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6292.119646] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6292.854708] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6293.570417] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6294.242582] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6294.936184] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6295.581786] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6296.227884] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6296.998310] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6297.729144] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6298.504919] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6299.145369] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6299.868919] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6300.536616] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6301.335507] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6302.021186] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6302.824754] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6303.561775] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6304.306214] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6305.016372] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6305.768799] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6306.474149] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6307.170304] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6307.911308] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6308.652954] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6309.353495] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6310.104688] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6310.888497] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6311.588628] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6312.217728] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6312.905771] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6313.480817] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6314.253863] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6314.972071] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6315.722641] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6316.453472] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6317.200200] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6317.947283] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6318.765412] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6319.419496] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6320.192654] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6320.922445] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6321.782865] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6322.454236] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6323.286328] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6324.012642] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6324.785933] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6325.513355] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6326.195815] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6326.935935] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6327.792658] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6328.544972] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6329.290490] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6329.951072] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6330.730407] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6331.495276] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6332.355359] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6333.096723] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6333.943508] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6334.686135] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6335.408574] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6336.151223] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6336.884071] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6337.685180] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6338.337604] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6339.011750] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6339.735043] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6340.438974] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6341.207970] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6341.944246] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6342.709203] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6343.552735] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6344.335607] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6345.050033] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6345.764645] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6346.460405] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6347.169455] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6347.888374] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6348.633370] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6349.377828] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6350.076896] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6350.774432] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6351.447406] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6352.145317] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6352.888700] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6353.608735] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6354.335744] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6354.996196] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6355.724430] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6356.448524] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6357.225353] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6357.902421] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6358.685712] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6359.499796] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6360.386805] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6361.162510] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6362.090834] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6363.011270] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6363.936615] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6364.735917] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6365.509657] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6366.329195] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6367.057657] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6367.794437] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6368.526190] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6369.437147] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6370.289685] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6371.009654] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6371.852382] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6372.639162] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6373.478503] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6374.340783] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6662.365096] LustreError: 164833:0:(ofd_dev.c:1780:ofd_create_hdl()) lustre-OST0000: unable to precreate: rc = -28 [ 6662.372266] LustreError: 163252:0:(osp_precreate.c:651:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -28 [ 6971.872705] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 6971.879491] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6971.889641] Lustre: Skipped 433 previous similar messages [ 6971.894184] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6972.905031] LustreError: lustre-OST0000-osc-MDT0000: operation ost_destroy to node 0@lo failed: rc = -107 [ 6972.907381] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6972.912601] LustreError: Skipped 5 previous similar messages [ 6974.945195] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6974.951287] Lustre: Skipped 1 previous similar message [ 6977.210162] LustreError: 217128:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 6977.332626] Lustre: server umount lustre-OST0000 complete [ 6980.073799] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6980.078280] Lustre: Skipped 1 previous similar message [ 6985.190416] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6985.193814] Lustre: Skipped 1 previous similar message [ 6985.887443] LustreError: 217329:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup [ 6985.891271] LustreError: 217329:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 6986.035707] Lustre: server umount lustre-MDT0000 complete [ 6988.112249] LustreError: 163213:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1769080740 with bad export cookie 11893446277821290833 [ 6988.115803] LustreError: MGC192.168.201.141@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6988.126753] LustreError: 163213:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 6988.220230] LustreError: 217532:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [ 6988.222069] LustreError: 217532:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 6988.302587] Lustre: server umount lustre-MDT0001 complete [ 6995.064081] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing unload_modules_local [ 6996.738869] Key type lgssc unregistered [ 6996.940645] LNet: 218063:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 6996.947134] LNetError: 218063:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 6996.963433] LNet: Removed LNI 192.168.201.141@tcp [ 6997.439158] Key type .llcrypt unregistered [ 6997.440229] Key type ._llcrypt unregistered [ 7012.597947] Key type ._llcrypt registered [ 7012.599854] Key type .llcrypt registered [ 7012.675254] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing load_modules_local [ 7013.154904] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 7013.187285] alg: No test for adler32 (adler32-zlib) [ 7014.124574] Lustre: Lustre: Build Version: 2.17.50_3_g69d656b [ 7014.262562] LNet: Added LNI 192.168.201.141@tcp [8/256/0/180] [ 7015.856822] Key type lgssc registered [ 7016.533926] Lustre: Echo OBD driver; http://www.lustre.org/ [ 7021.652699] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 7021.679086] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7023.029750] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7025.722780] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 7030.468588] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 7030.744728] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7032.975130] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 7035.971177] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 7038.044560] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 7041.817685] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 7042.019204] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7044.981184] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 7047.973392] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 7048.174214] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49666 to 0x280000401:49697) [ 7056.386970] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 7057.347840] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7057.355740] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 7057.364743] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7057.368775] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 7057.368811] LustreError: 219458:0:(osp_precreate.c:1263:osp_precreate_thread()) cfs_fail_timeout id 2107 sleeping for 20000ms [ 7060.451093] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 7060.451441] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7060.462874] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7060.466859] Lustre: Skipped 1 previous similar message [ 7064.574958] LustreError: 221835:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 7064.668584] Lustre: server umount lustre-OST0000 complete [ 7077.463129] LustreError: 219458:0:(osp_precreate.c:1263:osp_precreate_thread()) cfs_fail_timeout id 2107 awake [ 7094.638790] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 7094.848988] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7097.993873] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 7099.938384] LustreError: lustre-OST0000-osc-MDT0001: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7099.951014] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7099.952578] Lustre: lustre-OST0000-osc-MDT0001: Connection restored to 0@lo (at 0@lo) [ 7099.979961] Lustre: Skipped 1 previous similar message [ 7101.201585] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 7103.230625] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid in FULL state after 1 sec [ 7105.407373] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff945846d45800.ost_server_uuid 50 [ 7106.358363] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff945846d45800.ost_server_uuid in FULL state after 0 sec [ 7110.116171] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 7110.121524] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7110.127972] Lustre: Skipped 1 previous similar message [ 7110.132400] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7110.136966] Lustre: Skipped 1 previous similar message [ 7113.196929] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7113.199508] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7113.212161] Lustre: Skipped 1 previous similar message [ 7114.609078] LustreError: 223154:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 7114.613020] LustreError: 223154:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 7114.695406] Lustre: server umount lustre-OST0000 complete [ 7117.795442] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7117.803830] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7117.814406] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7123.229273] LustreError: 223358:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup [ 7123.231630] LustreError: 223358:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 7123.283139] Lustre: server umount lustre-MDT0000 complete [ 7123.428616] LustreError: 219438:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7123.439151] LustreError: 219438:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 7125.798174] LustreError: 223068:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1769080877 with bad export cookie 5264935870236703528 [ 7125.799802] LustreError: MGC192.168.201.141@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7125.806420] LustreError: 223068:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 7125.924405] LustreError: 223559:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [ 7125.928716] LustreError: 223559:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 7126.013486] Lustre: server umount lustre-MDT0001 complete [ 7131.898356] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing unload_modules_local [ 7133.489752] Key type lgssc unregistered [ 7133.694396] LNet: 224090:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 7133.699258] LNetError: 224090:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 7133.709740] LNet: Removed LNI 192.168.201.141@tcp [ 7134.222886] Key type .llcrypt unregistered [ 7134.225706] Key type ._llcrypt unregistered [ 7153.252123] Key type ._llcrypt registered [ 7153.254040] Key type .llcrypt registered [ 7153.343474] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing load_modules_local [ 7153.880756] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 7153.915065] alg: No test for adler32 (adler32-zlib) [ 7154.883385] Lustre: Lustre: Build Version: 2.17.50_3_g69d656b [ 7155.057612] LNet: Added LNI 192.168.201.141@tcp [8/256/0/180] [ 7156.696196] Key type lgssc registered [ 7157.450233] Lustre: Echo OBD driver; http://www.lustre.org/ [ 7163.075913] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 7163.086563] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7164.342944] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7166.555987] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 7171.737813] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 7172.022832] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7174.386151] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 7176.509588] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 7180.858687] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 7181.063248] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7184.086739] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 7188.556876] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 7188.594320] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 7188.671177] Lustre: lustre-OST0001: new disk, initializing [ 7188.674205] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 7188.720138] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 7189.746596] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49699 to 0x280000401:49729) [ 7190.546615] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 7190.551739] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 7190.616124] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 7192.069259] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 7199.855053] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7203.111789] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 7218.144244] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7218.146837] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7218.156219] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7221.218483] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7221.218611] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7221.221500] Lustre: Skipped 1 previous similar message [ 7221.226172] Lustre: Skipped 1 previous similar message [ 7222.359837] LustreError: 229135:0:(obd_class.h:479:obd_check_dev()) Device 28 not setup [ 7222.416494] Lustre: server umount lustre-MDT0000 complete [ 7224.374539] LustreError: 225758:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1769080976 with bad export cookie 11113799250732382758 [ 7224.376762] LustreError: MGC192.168.201.141@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7224.389633] LustreError: 225758:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 7224.511077] LustreError: 229337:0:(obd_class.h:479:obd_check_dev()) Device 27 not setup [ 7224.516300] LustreError: 229337:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 7224.605047] Lustre: server umount lustre-MDT0001 complete [ 7236.917134] LustreError: 229537:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 7236.920508] LustreError: 229537:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 7236.939628] Lustre: server umount lustre-OST0000 complete [ 7248.889778] LustreError: 229739:0:(obd_class.h:479:obd_check_dev()) Device 2 not setup [ 7248.892448] LustreError: 229739:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 7248.963850] Lustre: server umount lustre-OST0001 complete [ 7250.795749] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro [ 7250.820835] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 7251.306539] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro [ 7251.335852] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 7252.225791] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: ,errors=remount-ro [ 7252.280341] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 7252.825753] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: ,errors=remount-ro [ 7252.888842] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 7259.270354] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing load_modules_local [ 7264.071442] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7264.335589] LustreError: 231147:0:(ldlm_lib.c:1179:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7264.348138] LustreError: 231147:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 7264.389734] Lustre: mylustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7266.139633] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 7269.856269] LustreError: 231147:0:(ldlm_lib.c:1179:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7270.588236] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 7272.891327] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 7274.530870] Lustre: 232257:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 7278.221270] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 7278.449942] Lustre: mylustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7278.453580] Lustre: Skipped 1 previous similar message [ 7280.485759] LustreError: 232611:0:(ldlm_lib.c:1179:target_handle_connect()) mylustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7281.673989] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 7285.734502] LustreError: 232611:0:(ldlm_lib.c:1179:target_handle_connect()) mylustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7286.472771] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 7289.677719] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 7290.053694] Lustre: mylustre-OST0000: new connection from mylustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49731 to 0x280000401:49761) [ 7290.063446] Lustre: mylustre-OST0001: new connection from mylustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:33) [ 7294.114389] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7295.892924] Lustre: 234103:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 7310.826793] Lustre: mylustre-MDT0000-lwp-OST0001: Connection to mylustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7310.832154] Lustre: mylustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7310.856214] Lustre: Skipped 4 previous similar messages [ 7310.961822] Lustre: Skipped 5 previous similar messages [ 7315.583767] LustreError: 234412:0:(obd_class.h:479:obd_check_dev()) Device 10 not setup [ 7315.588336] LustreError: 234412:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 7315.652149] Lustre: server umount mylustre-MDT0000 complete [ 7315.937239] LustreError: 231147:0:(ldlm_lib.c:1179:target_handle_connect()) mylustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7315.962798] LustreError: 231147:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 7320.888951] LustreError: 232259:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1769081072 with bad export cookie 11113799250732385523 [ 7320.910932] LustreError: MGC192.168.201.141@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7321.063752] Lustre: mylustre-MDT0001-lwp-OST0000: Connection to mylustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7321.074250] Lustre: mylustre-MDT0001: Not available for connect from 0@lo (stopping) [ 7321.075594] Lustre: Skipped 1 previous similar message [ 7321.226531] Lustre: server umount mylustre-MDT0001 complete [ 7328.351883] LustreError: 234816:0:(obd_class.h:479:obd_check_dev()) Device 23 not setup [ 7328.354716] LustreError: 234816:0:(obd_class.h:479:obd_check_dev()) Skipped 15 previous similar messages [ 7328.803883] Lustre: server umount mylustre-OST0000 complete [ 7334.879464] Lustre: server umount mylustre-OST0001 complete [ 7339.158113] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro [ 7339.246226] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 7340.083734] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro [ 7340.187679] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 7341.937471] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: ,errors=remount-ro [ 7342.038964] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 7343.648776] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: ,errors=remount-ro [ 7343.823179] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 7359.779311] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing load_modules_local [ 7368.990262] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7369.562116] LustreError: 236430:0:(ldlm_lib.c:1179:target_handle_connect()) tfs-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7369.596989] LustreError: 236430:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 7369.695426] Lustre: tfs-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7369.708130] Lustre: Skipped 1 previous similar message [ 7373.587696] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 7381.008579] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 7385.386655] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 7387.970317] Lustre: 237540:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 7393.509317] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 7394.856522] LustreError: 237895:0:(ldlm_lib.c:1179:target_handle_connect()) tfs-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7394.879540] LustreError: 237895:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 7394.893130] Lustre: tfs-OST0000: new connection from tfs-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49731 to 0x280000401:49793) [ 7398.472979] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 7405.708841] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 7405.861182] Lustre: tfs-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 7405.867080] Lustre: Skipped 2 previous similar messages [ 7406.906426] Lustre: tfs-OST0001: new connection from tfs-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:35 to 0x2c0000401:65) [ 7410.865838] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 7416.739471] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7419.556218] Lustre: 239387:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 7432.674433] Lustre: tfs-MDT0000-osp-MDT0001: Connection to tfs-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7432.678187] Lustre: tfs-MDT0000: Not available for connect from 0@lo (stopping) [ 7432.685775] LustreError: tfs-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7432.688053] Lustre: Skipped 1 previous similar message [ 7432.700177] Lustre: Skipped 4 previous similar messages [ 7436.405545] LustreError: 239637:0:(obd_class.h:479:obd_check_dev()) Device 10 not setup [ 7436.411089] LustreError: 239637:0:(obd_class.h:479:obd_check_dev()) Skipped 9 previous similar messages [ 7436.498937] Lustre: server umount tfs-MDT0000 complete [ 7437.797928] LustreError: 236430:0:(ldlm_lib.c:1179:target_handle_connect()) tfs-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7437.811129] LustreError: 236430:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 5 previous similar messages [ 7439.559680] LustreError: 239389:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1769081191 with bad export cookie 11113799250732387413 [ 7439.563233] LustreError: MGC192.168.201.141@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7439.569781] LustreError: 239389:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 6 previous similar messages [ 7439.828711] Lustre: server umount tfs-MDT0001 complete [ 7453.315874] Lustre: server umount tfs-OST0000 complete [ 7466.839627] Lustre: server umount tfs-OST0001 complete [ 7469.411905] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro [ 7469.488353] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 7470.470986] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro [ 7470.581408] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 7471.579308] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: ,errors=remount-ro [ 7471.658164] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 7472.522837] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: ,errors=remount-ro [ 7472.598721] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 7483.032992] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing load_modules_local [ 7490.767673] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7491.233389] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7495.140087] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 7501.780726] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 7505.170233] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 7507.188315] Lustre: 242760:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 7512.571533] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 7515.956110] LustreError: 243119:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7515.966984] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49731 to 0x280000401:49825) [ 7515.974628] LustreError: 243119:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 7517.376953] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 7523.478729] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 7525.324882] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:35 to 0x2c0000401:97) [ 7528.447250] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 7535.136858] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7537.920879] Lustre: 244607:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 7548.751853] Lustre: DEBUG MARKER: == conf-sanity test 104a: Make sure user defined options are reflected in mount ========================================================== 06:28:19 (1769081299) [ 7563.744080] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7563.749995] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7563.765473] Lustre: Skipped 2 previous similar messages [ 7563.770756] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7569.639436] LustreError: 244978:0:(obd_class.h:479:obd_check_dev()) Device 10 not setup [ 7569.646136] LustreError: 244978:0:(obd_class.h:479:obd_check_dev()) Skipped 25 previous similar messages [ 7569.811564] Lustre: server umount lustre-MDT0000 complete [ 7573.117290] LustreError: 241625:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1769081325 with bad export cookie 11113799250732389072 [ 7573.120879] LustreError: MGC192.168.201.141@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7573.127156] LustreError: 241625:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 1 previous similar message [ 7579.492449] Lustre: server umount lustre-MDT0001 complete [ 7597.023176] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 7597.112104] Lustre: server umount lustre-OST0000 complete [ 7605.989217] Lustre: server umount lustre-OST0001 complete [ 7611.032522] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_hostid [ 7621.150769] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing load_modules_local [ 7630.183714] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 7635.266508] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 7640.745715] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 7646.668444] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 7652.142080] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 7652.182478] LDISKFS-fs (dm-0): Mount option "noacl" will be removed by 3.5 [ 7652.182478] Contact linux-ldiskfs@vger.kernel.org if you think we should keep it. [ 7652.182478] [ 7652.195065] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: acl,user_xattr,errors=remount-ro,noacl,no_mbcache,nodelalloc [ 7652.343938] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 7652.370374] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 7652.421053] Lustre: lustre-MDT0000: new disk, initializing [ 7652.475114] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7652.480351] Lustre: Skipped 3 previous similar messages [ 7652.498290] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 7655.783945] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 7664.875750] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 7664.996690] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 7665.099887] Lustre: 247770:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 7665.192215] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 7665.195728] Lustre: Skipped 1 previous similar message [ 7665.268439] Lustre: lustre-MDT0001: new disk, initializing [ 7665.364723] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 7665.378962] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 7669.065305] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 7678.566889] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 7678.621696] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 7678.752526] Lustre: lustre-OST0000: new disk, initializing [ 7678.755617] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 7680.268636] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 7680.273546] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 7680.390914] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 7684.047846] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 7692.817790] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 7692.901779] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 7692.997786] Lustre: lustre-OST0001: new disk, initializing [ 7693.004858] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 7694.538858] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 7694.549795] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 7694.630589] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 7698.185926] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 7708.033212] Lustre: DEBUG MARKER: == conf-sanity test 104b: Mount uses last flock argument ========================================================== 06:30:58 (1769081458) [ 7720.036549] Lustre: DEBUG MARKER: == conf-sanity test 105: check file creation for ro and rw bind mnt pt ========================================================== 06:31:10 (1769081470) [ 7735.787164] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7735.806763] Lustre: Skipped 5 previous similar messages [ 7735.812043] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7735.822836] Lustre: Skipped 9 previous similar messages [ 7740.080794] LustreError: 251199:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 7740.083582] LustreError: 251199:0:(obd_class.h:479:obd_check_dev()) Skipped 25 previous similar messages [ 7740.253603] Lustre: server umount lustre-OST0000 complete [ 7740.901370] LustreError: 249448:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7740.920574] LustreError: 249448:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 6 previous similar messages [ 7749.782930] Lustre: server umount lustre-MDT0000 complete [ 7753.088847] LustreError: 247762:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1769081505 with bad export cookie 11113799250732390633 [ 7753.096817] LustreError: 247762:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 1 previous similar message [ 7753.102640] LustreError: MGC192.168.201.141@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7759.459519] Lustre: server umount lustre-MDT0001 complete [ 7768.791209] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing unload_modules_local [ 7784.708648] Lustre: server umount lustre-OST0001 complete [ 7790.417380] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_hostid [ 7804.075911] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing load_modules_local [ 7812.781484] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 7818.674735] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 7823.940804] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 7829.141942] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 7835.844693] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 7835.913446] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7836.144986] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 7836.174088] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 7836.242725] Lustre: lustre-MDT0000: new disk, initializing [ 7836.353603] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 7840.471743] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 7849.688157] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 7849.761722] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 7849.822762] Lustre: 254677:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 7850.127096] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 7853.827352] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 7859.677198] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 7862.363692] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 7866.936989] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 7866.990893] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 7867.173700] Lustre: lustre-OST0000: new disk, initializing [ 7867.176068] Lustre: Skipped 1 previous similar message [ 7867.179466] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 7867.184807] Lustre: Skipped 2 previous similar messages [ 7868.925558] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 7868.932057] Lustre: Skipped 1 previous similar message [ 7868.936814] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 7869.031899] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 7871.605919] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 7876.372901] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 7884.261454] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 7884.261623] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7884.264461] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7884.264468] Lustre: Skipped 5 previous similar messages [ 7884.271074] LustreError: Skipped 1 previous similar message [ 7884.282950] Lustre: Skipped 6 previous similar messages [ 7887.290287] LustreError: 257117:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 7887.300613] LustreError: 257117:0:(obd_class.h:479:obd_check_dev()) Skipped 25 previous similar messages [ 7887.425441] Lustre: server umount lustre-OST0000 complete [ 7890.915713] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7904.223564] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 7904.345540] Lustre: server umount lustre-MDT0000 complete [ 7906.837066] LustreError: 254668:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1769081658 with bad export cookie 11113799250732392103 [ 7906.837735] LustreError: MGC192.168.201.141@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7906.843680] LustreError: 254668:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 7907.091037] Lustre: server umount lustre-MDT0001 complete [ 7913.253302] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing unload_modules_local [ 7914.855298] Key type lgssc unregistered [ 7915.047745] LNet: 258053:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 7915.051317] LNetError: 258053:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 7915.063129] LNet: Removed LNI 192.168.201.141@tcp [ 7915.544237] Key type .llcrypt unregistered [ 7915.545345] Key type ._llcrypt unregistered [ 7930.278483] Key type ._llcrypt registered [ 7930.282022] Key type .llcrypt registered [ 7930.330837] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing load_modules_local [ 7930.755709] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 7930.787821] alg: No test for adler32 (adler32-zlib) [ 7931.695201] Lustre: Lustre: Build Version: 2.17.50_3_g69d656b [ 7931.808259] LNet: Added LNI 192.168.201.141@tcp [8/256/0/180] [ 7933.399172] Key type lgssc registered [ 7934.092453] Lustre: Echo OBD driver; http://www.lustre.org/ [ 7938.520096] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 7938.530942] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7939.811985] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7942.190150] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 7947.208959] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 7947.448337] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7950.001800] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 7953.163770] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 7955.024996] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 7958.915786] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 7959.154062] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7962.435492] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 7964.277262] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:3 to 0x280000401:33) [ 7965.247698] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 7965.784929] Lustre: Setting parameter lustre-OST0000.ost.unknown_param=50 in log lustre-OST0000 [ 7969.764755] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7969.774346] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 7969.777932] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7972.833450] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7972.835813] Lustre: Skipped 1 previous similar message [ 7973.043110] LustreError: 261763:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 7973.092694] Lustre: server umount lustre-OST0000 complete [ 7977.952689] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7977.953507] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7977.955873] Lustre: Skipped 1 previous similar message [ 7977.959019] Lustre: Skipped 1 previous similar message [ 7981.138214] LustreError: 261974:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup [ 7981.141010] LustreError: 261974:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 7981.172165] Lustre: server umount lustre-MDT0000 complete [ 7983.073481] LustreError: 259472:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7983.082596] LustreError: 259472:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 7983.150714] LustreError: 261092:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1769081735 with bad export cookie 8009300018192240884 [ 7983.152195] LustreError: MGC192.168.201.141@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7983.155934] LustreError: 261092:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 7983.288070] LustreError: 262175:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [ 7983.292006] LustreError: 262175:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 7983.413599] Lustre: server umount lustre-MDT0001 complete [ 7989.449608] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing unload_modules_local [ 7991.214491] Key type lgssc unregistered [ 7991.447544] LNet: 262707:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 7991.454666] LNetError: 262707:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 7991.468971] LNet: Removed LNI 192.168.201.141@tcp [ 7992.049983] Key type .llcrypt unregistered [ 7992.051329] Key type ._llcrypt unregistered [ 8002.440884] Key type ._llcrypt registered [ 8002.442461] Key type .llcrypt registered [ 8002.509518] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing load_modules_local [ 8003.023268] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8003.055171] alg: No test for adler32 (adler32-zlib) [ 8004.009887] Lustre: Lustre: Build Version: 2.17.50_3_g69d656b [ 8004.151558] LNet: Added LNI 192.168.201.141@tcp [8/256/0/180] [ 8005.743203] Key type lgssc registered [ 8006.435471] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8012.715626] Lustre: lustre-OST0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 8012.728561] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 8028.191624] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 8033.311228] LustreError: 263859:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.201.141@tcp: failed processing log, type 4: rc = -110 [ 8059.935238] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8062.553605] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 8065.176264] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 8066.270713] LustreError: 264523:0:(obd_class.h:479:obd_check_dev()) Device 1 not setup [ 8066.324236] Lustre: server umount lustre-OST0000 complete [ 8074.687828] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing unload_modules_local [ 8076.075776] Key type lgssc unregistered [ 8076.221097] LNet: 265358:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8076.224687] LNetError: 265358:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8076.237364] LNet: Removed LNI 192.168.201.141@tcp [ 8076.655148] Key type .llcrypt unregistered [ 8076.656746] Key type ._llcrypt unregistered [ 8096.308755] Key type ._llcrypt registered [ 8096.310377] Key type .llcrypt registered [ 8096.353544] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing load_modules_local [ 8096.840081] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8096.849977] alg: No test for adler32 (adler32-zlib) [ 8097.800969] Lustre: Lustre: Build Version: 2.17.50_3_g69d656b [ 8097.940755] LNet: Added LNI 192.168.201.141@tcp [8/256/0/180] [ 8099.536323] Key type lgssc registered [ 8100.123704] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8106.957768] loop0: detected capacity change from 0 to 204800000 [ 8107.083374] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8107.093584] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8107.256123] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8107.286832] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 8107.886481] loop0: detected capacity change from 0 to 204800000 [ 8107.996226] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8108.004158] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8108.209249] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8108.237181] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 8108.837326] loop0: detected capacity change from 0 to 204800000 [ 8108.975619] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8108.981382] blk_update_request: operation not supported error, dev loop0, sector 8224 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8109.034182] blk_update_request: operation not supported error, dev loop0, sector 58264 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8109.096653] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 8109.631543] loop0: detected capacity change from 0 to 204800000 [ 8109.768413] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8109.928866] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 8110.411856] loop0: detected capacity change from 0 to 204800000 [ 8110.427220] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 8112.281989] loop0: detected capacity change from 0 to 204800000 [ 8112.293710] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 8114.023972] loop0: detected capacity change from 0 to 204800000 [ 8114.039352] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 8115.821487] loop0: detected capacity change from 0 to 204800000 [ 8115.834735] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 8117.512020] loop0: detected capacity change from 0 to 204800000 [ 8117.552469] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 8117.585669] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 8117.598588] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8117.632305] Lustre: 268733:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 8117.635646] Lustre: 268733:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index 'nodemap' with [0x200000003:0x3:0x0]: rc = 0 [ 8117.641163] Lustre: 268733:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000003:0xc:0x0]: rc = 0 [ 8117.649368] Lustre: 268733:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000003:0xb:0x0]: rc = 0 [ 8117.654261] Lustre: 268733:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x2010000-MDT0000' with [0x200000005:0x3:0x0]: rc = 0 [ 8117.659920] Lustre: 268733:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000003:0xa:0x0]: rc = 0 [ 8117.666482] Lustre: 268733:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x1010000-MDT0000' with [0x200000005:0x2:0x0]: rc = 0 [ 8117.673500] Lustre: 268733:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x10000-MDT0000' with [0x200000005:0x1:0x0]: rc = 0 [ 8117.680286] Lustre: 268733:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x2020000' with [0x200000006:0x2020000:0x0]: rc = 0 [ 8117.687215] Lustre: 268733:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x20000' with [0x200000006:0x20000:0x0]: rc = 0 [ 8117.694170] Lustre: 268733:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x1020000' with [0x200000006:0x1020000:0x0]: rc = 0 [ 8117.699729] Lustre: 268733:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000006:0x2010000:0x0]: rc = 0 [ 8117.705407] Lustre: 268733:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000006:0x1010000:0x0]: rc = 0 [ 8117.709746] Lustre: 268733:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000006:0x10000:0x0]: rc = 0 [ 8120.854217] LustreError: 269001:0:(obd_class.h:479:obd_check_dev()) Device 2 not setup [ 8120.931111] Lustre: server umount MGS complete [ 8121.315539] loop0: detected capacity change from 0 to 204800000 [ 8121.369312] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8121.603814] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8121.639204] Lustre: lustre-MDT0000: Aborting client recovery [ 8122.152174] loop1: detected capacity change from 0 to 204800000 [ 8122.205508] LDISKFS-fs (loop1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 8122.246429] LDISKFS-fs (loop1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 8122.279410] Lustre: 269168:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 8122.284467] Lustre: 269168:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0001: restore index '0x10000-MDT0001' with [0x200000005:0x1:0x0]: rc = 0 [ 8122.288491] Lustre: 269168:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0001: restore index '0x2010000-MDT0001' with [0x200000005:0x3:0x0]: rc = 0 [ 8122.294221] Lustre: 269168:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0001: restore index '0x2010000' with [0x200000003:0x6:0x0]: rc = 0 [ 8122.301201] Lustre: 269168:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0001: restore index '0x10000' with [0x200000003:0x4:0x0]: rc = 0 [ 8122.306711] Lustre: 269168:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0001: restore index '0x1010000' with [0x200000003:0x5:0x0]: rc = 0 [ 8122.313530] Lustre: 269168:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0001: restore index '0x1010000-MDT0001' with [0x200000005:0x2:0x0]: rc = 0 [ 8122.323502] Lustre: Found index 1 for lustre-MDT0001, updating log [ 8122.436658] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8122.464118] Lustre: lustre-MDT0001: Aborting client recovery [ 8122.936961] loop2: detected capacity change from 0 to 204800000 [ 8122.973659] LDISKFS-fs (loop2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 8123.022178] LDISKFS-fs (loop2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 8123.047547] Lustre: 269251:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 8123.052917] Lustre: 269251:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0000: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 8123.058678] Lustre: 269251:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0000: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 8123.065206] Lustre: 269251:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0000: restore index '0x1020000-OST0000' with [0x200000005:0x2:0x0]: rc = 0 [ 8123.069676] Lustre: 269251:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0000: restore index '0x2020000-OST0000' with [0x200000005:0x3:0x0]: rc = 0 [ 8123.074183] Lustre: 269251:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0000: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 8123.079222] Lustre: 269251:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0000: restore index '0x20000-OST0000' with [0x200000005:0x1:0x0]: rc = 0 [ 8123.138572] Lustre: Found index 0 for lustre-OST0000, updating log [ 8123.187673] Lustre: lustre-OST0000: Aborting recovery [ 8123.614879] loop3: detected capacity change from 0 to 204800000 [ 8123.674482] LDISKFS-fs (loop3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 8123.708070] LDISKFS-fs (loop3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 8123.734772] Lustre: 269340:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 8123.743508] Lustre: 269340:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0001: restore index '0x20000-OST0001' with [0x200000005:0x1:0x0]: rc = 0 [ 8123.749278] Lustre: 269340:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0001: restore index '0x2020000-OST0001' with [0x200000005:0x3:0x0]: rc = 0 [ 8123.758579] Lustre: 269340:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0001: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 8123.764320] Lustre: 269340:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0001: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 8123.768513] Lustre: 269340:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0001: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 8123.772715] Lustre: 269340:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0001: restore index '0x1020000-OST0001' with [0x200000005:0x2:0x0]: rc = 0 [ 8123.821875] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 8123.824127] Lustre: Skipped 1 previous similar message [ 8123.832159] Lustre: lustre-OST0001: Aborting recovery [ 8129.049533] LustreError: 269111:0:(lod_dev.c:510:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 8, retries 0, failed: rc = -108 [ 8131.055663] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 8131.056724] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 8131.090187] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000bd0 [ 8131.097688] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000bd0 [ 8132.068833] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:5 to 0x280000400:33) [ 8132.079325] Lustre: lustre-OST0001: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:5 to 0x2c0000400:33) [ 8136.120845] LustreError: 269079:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0000: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0000 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0000/identity_upcall, time 270us: rc = -2 [ 8147.940395] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8147.945509] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8147.951210] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8152.546882] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8152.547203] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8152.561877] Lustre: Skipped 2 previous similar messages [ 8152.572225] Lustre: Skipped 2 previous similar messages [ 8153.742478] LustreError: 269692:0:(obd_class.h:479:obd_check_dev()) Device 10 not setup [ 8153.744949] LustreError: 269692:0:(obd_class.h:479:obd_check_dev()) Skipped 1 previous similar message [ 8153.798148] Lustre: server umount lustre-MDT0000 complete [ 8154.273290] LustreError: 269063:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1769081906 with bad export cookie 793881192718996265 [ 8154.277310] LustreError: MGC192.168.201.141@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8154.281923] LustreError: 269063:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 1 previous similar message [ 8157.663657] LustreError: 269281:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8157.665612] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8157.667624] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 8157.667632] Lustre: Skipped 1 previous similar message [ 8157.680928] LustreError: 269281:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 8157.685873] Lustre: Skipped 1 previous similar message [ 8159.712492] LustreError: 269745:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8159.712753] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 8159.718581] LustreError: 269745:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 8159.724147] Lustre: Skipped 2 previous similar messages [ 8160.390196] LustreError: 269743:0:(obd_class.h:479:obd_check_dev()) Device 18 not setup [ 8160.392534] LustreError: 269743:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 8160.477253] Lustre: server umount lustre-MDT0001 complete [ 8167.093071] LustreError: 269795:0:(obd_class.h:479:obd_check_dev()) Device 23 not setup [ 8167.099144] LustreError: 269795:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 8167.168535] Lustre: server umount lustre-OST0000 complete [ 8173.749206] LustreError: 269846:0:(obd_class.h:479:obd_check_dev()) Device 2 not setup [ 8173.754150] LustreError: 269846:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 8173.816684] Lustre: server umount lustre-OST0001 complete [ 8178.028614] Lustre: DEBUG MARKER: == conf-sanity test 109a: test lctl clear_conf fsname ==== 06:38:49 (1769081929) [ 8191.835101] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_hostid [ 8196.829968] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing load_modules_local [ 8201.564277] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 8204.530468] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 8207.686287] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 8210.792725] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 8213.693688] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 8213.721569] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8213.823339] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 8213.839250] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 8213.879842] Lustre: lustre-MDT0000: new disk, initializing [ 8213.911983] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8213.923194] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 8215.493890] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 8220.304951] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 8220.330547] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 8220.370545] Lustre: 273392:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 8220.398908] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 8220.401080] Lustre: Skipped 1 previous similar message [ 8220.439494] Lustre: lustre-MDT0001: new disk, initializing [ 8220.474610] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8220.490706] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 8220.495086] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 8221.981495] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 8225.590384] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 8227.044847] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 8229.868596] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 8229.907881] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 8230.029158] Lustre: lustre-OST0000: new disk, initializing [ 8230.031536] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 8230.067888] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8232.599310] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 8236.533905] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 8237.053766] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 8237.058177] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 8237.105768] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 8244.574395] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [ 8252.997737] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [ 8257.847971] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [ 8293.344910] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 8293.351898] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8293.360066] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8293.362309] Lustre: Skipped 1 previous similar message [ 8303.585673] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8303.588718] Lustre: Skipped 3 previous similar messages [ 8307.167100] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 8307.185431] LustreError: 277006:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 8307.187861] LustreError: 277006:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 8307.229607] Lustre: server umount lustre-OST0000 complete [ 8308.704965] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8308.711084] Lustre: Skipped 2 previous similar messages [ 8314.781179] Lustre: server umount lustre-MDT0000 complete [ 8316.266452] LustreError: 274171:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1769082068 with bad export cookie 793881192718998967 [ 8316.269861] LustreError: MGC192.168.201.141@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8316.270812] LustreError: 274171:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 8316.415951] Lustre: server umount lustre-MDT0001 complete [ 8319.536954] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8321.061706] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 8322.931446] Lustre: server umount MGS complete [ 8327.323724] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8327.486676] LustreError: 278973:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8327.494186] LustreError: 278985:0:(qmt_pool.c:1523:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [ 8327.521776] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8328.974832] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 8332.332311] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 8333.887833] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 8335.793961] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 8337.030173] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 8339.477342] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 8341.722863] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 8343.703301] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 8377.826292] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8377.828900] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8377.829427] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 8377.829432] LustreError: Skipped 1 previous similar message [ 8377.830843] Lustre: Skipped 1 previous similar message [ 8377.841609] Lustre: Skipped 5 previous similar messages [ 8381.999136] LustreError: 281654:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 8382.001772] LustreError: 281654:0:(obd_class.h:479:obd_check_dev()) Skipped 19 previous similar messages [ 8382.056401] Lustre: server umount lustre-OST0000 complete [ 8383.967828] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8389.659076] Lustre: server umount lustre-MDT0000 complete [ 8391.157466] LustreError: 279000:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1769082143 with bad export cookie 793881192719000780 [ 8391.160534] LustreError: MGC192.168.201.141@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8391.162588] LustreError: 279000:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 8391.297265] Lustre: server umount lustre-MDT0001 complete [ 8395.415808] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing unload_modules_local [ 8396.685266] Key type lgssc unregistered [ 8396.833575] LNet: 282587:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8396.837791] LNetError: 282587:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8396.848717] LNet: Removed LNI 192.168.201.141@tcp [ 8397.230198] Key type .llcrypt unregistered [ 8397.231719] Key type ._llcrypt unregistered [ 8413.114738] Key type ._llcrypt registered [ 8413.116981] Key type .llcrypt registered [ 8413.193412] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_hostid [ 8420.526064] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing load_modules_local [ 8420.969501] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8420.976395] alg: No test for adler32 (adler32-zlib) [ 8421.856189] Lustre: Lustre: Build Version: 2.17.50_3_g69d656b [ 8421.949969] LNet: Added LNI 192.168.201.141@tcp [8/256/0/180] [ 8423.543238] Key type lgssc registered [ 8423.993776] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8427.701056] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 8430.642962] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 8433.494183] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 8436.135052] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 8439.295174] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 8439.320314] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 8439.327488] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8440.435662] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 8440.452607] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 8440.496504] Lustre: lustre-MDT0000: new disk, initializing [ 8440.535705] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8440.545364] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 8442.265470] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 8447.108279] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 8447.141698] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 8447.180253] Lustre: 286492:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 8447.208148] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 8447.210851] Lustre: Skipped 1 previous similar message [ 8447.251058] Lustre: lustre-MDT0001: new disk, initializing [ 8447.294905] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8447.318223] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 8447.326228] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 8448.980829] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 8452.653312] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 8454.177145] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 8457.265065] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 8457.289693] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 8457.403425] Lustre: lustre-OST0000: new disk, initializing [ 8457.405236] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 8457.440110] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8459.712710] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 8460.862689] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 8460.866101] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 8460.916956] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 8463.472501] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 8467.340501] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [ 8473.565716] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [ 8481.763270] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [ 8488.503650] Lustre: Modifying parameter lustre-client.llite.max_read_ahead_mb=64 in log lustre-client [ 8525.793936] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8525.795118] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8525.799961] Lustre: Skipped 1 previous similar message [ 8530.734246] LustreError: 290103:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 8530.781820] Lustre: server umount lustre-OST0000 complete [ 8534.495977] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8534.499289] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8534.506474] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8534.508823] Lustre: Skipped 1 previous similar message [ 8536.033215] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8536.033685] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8538.376757] LustreError: 290304:0:(obd_class.h:479:obd_check_dev()) Device 22 not setup [ 8538.379302] LustreError: 290304:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 8538.415538] Lustre: server umount lustre-MDT0000 complete [ 8539.976962] LustreError: 289777:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1769082292 with bad export cookie 7346378888303969564 [ 8539.979423] LustreError: MGC192.168.201.141@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8539.981724] LustreError: 289777:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 8540.054577] LustreError: 290505:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 8540.057896] LustreError: 290505:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 8540.141549] Lustre: server umount lustre-MDT0001 complete [ 8544.037210] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8545.640494] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 8547.651344] LustreError: 291445:0:(obd_class.h:479:obd_check_dev()) Device 2 not setup [ 8547.654254] LustreError: 291445:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 8547.679879] Lustre: server umount MGS complete [ 8552.313046] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8552.564691] LustreError: 292080:0:(qmt_pool.c:1523:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [ 8552.601890] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8554.561672] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 8558.049319] LustreError: 292068:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8558.055180] LustreError: 292068:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 8558.204705] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 8558.409033] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8560.120344] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 8562.949373] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 8564.824833] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 8567.987926] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 8568.143894] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8570.724624] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 8572.981525] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 8597.987788] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8597.990056] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8597.991910] Lustre: Skipped 1 previous similar message [ 8597.994556] Lustre: Skipped 1 previous similar message [ 8597.997084] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 8602.159134] LustreError: 294725:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 8602.163036] LustreError: 294725:0:(obd_class.h:479:obd_check_dev()) Skipped 1 previous similar message [ 8602.218335] Lustre: server umount lustre-OST0000 complete [ 8604.642285] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8604.644747] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8604.649611] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8604.653220] Lustre: Skipped 1 previous similar message [ 8610.179516] LustreError: 294927:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup [ 8610.182196] LustreError: 294927:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 8610.234759] Lustre: server umount lustre-MDT0000 complete [ 8611.970153] LustreError: 293688:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1769082364 with bad export cookie 7346378888303971405 [ 8611.972962] LustreError: MGC192.168.201.141@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8611.979770] LustreError: 293688:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 8612.125438] Lustre: server umount lustre-MDT0001 complete [ 8616.947380] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing unload_modules_local [ 8618.321049] Key type lgssc unregistered [ 8618.494969] LNet: 295661:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8618.500452] LNetError: 295661:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8619.558078] LNet: Removed LNI 192.168.201.141@tcp [ 8619.967903] Key type .llcrypt unregistered [ 8619.971253] Key type ._llcrypt unregistered [ 8629.041066] Key type ._llcrypt registered [ 8629.042652] Key type .llcrypt registered [ 8629.093848] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_hostid [ 8636.079475] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing load_modules_local [ 8636.438894] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8636.448241] alg: No test for adler32 (adler32-zlib) [ 8637.337272] Lustre: Lustre: Build Version: 2.17.50_3_g69d656b [ 8637.439735] LNet: Added LNI 192.168.201.141@tcp [8/256/0/180] [ 8639.031160] Key type lgssc registered [ 8639.436146] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8642.742438] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 8644.966402] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 8648.295353] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing load_modules_local [ 8651.485859] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 8654.423581] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 8654.459558] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 8654.467373] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8655.579689] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 8655.596277] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 8655.638357] Lustre: lustre-MDT0000: new disk, initializing [ 8655.674652] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8655.683501] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 8657.194264] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 8661.834477] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 8661.861248] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 8661.969468] Lustre: lustre-OST0000: new disk, initializing [ 8661.973514] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 8661.976766] Lustre: Skipped 1 previous similar message [ 8662.006834] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8664.149754] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 8665.475109] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 8665.481217] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 8665.492810] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 8977.889752] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8977.894986] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8980.292401] LustreError: 300589:0:(obd_class.h:479:obd_check_dev()) Device 13 not setup [ 8980.451974] Lustre: server umount lustre-MDT0000 complete [ 8981.923075] LustreError: 298824:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1769082734 with bad export cookie 16270998719827426456 [ 8981.923476] LustreError: MGC192.168.201.141@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8981.927377] LustreError: 298824:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 8981.957322] LustreError: 300794:0:(obd_class.h:479:obd_check_dev()) Device 2 not setup [ 8981.959355] LustreError: 300794:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 8982.013642] Lustre: server umount lustre-OST0000 complete [ 8991.434309] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_hostid [ 8995.200881] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing load_modules_local [ 8999.094074] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 9001.625829] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 9003.900762] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 9006.381489] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 9008.903044] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 9008.928969] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9009.022313] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 9009.037408] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 9009.075763] Lustre: lustre-MDT0000: new disk, initializing [ 9009.104098] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9009.112173] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 9010.659136] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 9015.175543] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 9015.204876] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 9015.250202] Lustre: 303833:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 9015.282843] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 9015.286978] Lustre: Skipped 1 previous similar message [ 9015.331001] Lustre: lustre-MDT0001: new disk, initializing [ 9015.369948] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9015.386151] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 9015.391316] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 9016.771264] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 9020.016254] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 9021.297948] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 9023.730996] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 9023.771374] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 9023.873972] Lustre: lustre-OST0000: new disk, initializing [ 9023.876518] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 9023.909880] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9025.017691] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 9025.022082] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 9025.056096] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 9025.939826] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 9029.152774] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 9030.541963] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 9030.617995] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 9032.067990] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 9032.157064] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 9035.233667] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 9035.233740] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9035.236461] LustreError: Skipped 1 previous similar message [ 9035.241071] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9035.244948] Lustre: Skipped 1 previous similar message [ 9038.958082] LustreError: 306771:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 9038.960974] LustreError: 306771:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 9038.997148] Lustre: server umount lustre-OST0000 complete [ 9040.864029] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 9040.868331] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9040.874910] Lustre: Skipped 1 previous similar message [ 9040.877729] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9045.472233] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9045.472697] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9046.591508] LustreError: 306972:0:(obd_class.h:479:obd_check_dev()) Device 22 not setup [ 9046.593826] LustreError: 306972:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 9046.625353] Lustre: server umount lustre-MDT0000 complete [ 9047.860481] LustreError: 303823:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1769082799 with bad export cookie 16270998719830367233 [ 9047.862287] LustreError: MGC192.168.201.141@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9047.865088] LustreError: 303823:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 9047.980126] Lustre: server umount lustre-MDT0001 complete [ 9049.151510] Lustre: DEBUG MARKER: SKIP: conf-sanity test_111 skipping SLOW test 111 [ 9049.715896] Lustre: DEBUG MARKER: == conf-sanity test 112a: mount OST with no_create option ========================================================== 06:53:21 (1769082801) [ 9052.095700] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9052.290348] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9053.598453] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 9056.729993] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 9058.247385] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 9060.116820] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 9061.352572] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 9063.664963] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 9065.732743] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 9067.554201] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 9070.046618] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 9070.072630] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 9070.131940] Lustre: lustre-OST0001: new disk, initializing [ 9070.134141] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 9070.165882] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 9070.168471] Lustre: Skipped 2 previous similar messages [ 9072.277434] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 9073.088588] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 9073.092073] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 9073.107502] LustreError: 310691:0:(osp_precreate.c:969:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [ 9073.112798] LustreError: 310691:0:(osp_precreate.c:969:osp_precreate_cleanup_orphans()) Skipped 1 previous similar message [ 9074.146400] LustreError: 310691:0:(osp_precreate.c:969:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [ 9074.149348] LustreError: 310691:0:(osp_precreate.c:969:osp_precreate_cleanup_orphans()) Skipped 1 previous similar message [ 9075.170554] LustreError: 310696:0:(osp_precreate.c:969:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [ 9075.174220] LustreError: 310696:0:(osp_precreate.c:969:osp_precreate_cleanup_orphans()) Skipped 1 previous similar message [ 9076.182681] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 9077.223521] LustreError: 310696:0:(osp_precreate.c:969:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [ 9077.228658] LustreError: 310696:0:(osp_precreate.c:969:osp_precreate_cleanup_orphans()) Skipped 3 previous similar messages [ 9077.301536] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 1 sec [ 9078.598542] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff945850751800.ost_server_uuid 50 [ 9079.164754] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff945850751800.ost_server_uuid in FULL state after 0 sec [ 9080.570219] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff945850751800.ost_server_uuid 50 [ 9081.173838] Lustre: DEBUG MARKER: osc.lustre-OST0001-osc-ffff945850751800.ost_server_uuid in FULL state after 0 sec [ 9081.314684] LustreError: 310696:0:(osp_precreate.c:969:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [ 9081.318919] LustreError: 310696:0:(osp_precreate.c:969:osp_precreate_cleanup_orphans()) Skipped 7 previous similar messages [ 9087.464881] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 9095.622675] Lustre: lustre-OST0001: Not available for connect from 192.168.201.41@tcp (stopping) [ 9095.624850] Lustre: Skipped 1 previous similar message [ 9097.697514] LustreError: lustre-OST0001-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 9097.697752] Lustre: lustre-OST0001-osc-MDT0000: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9097.701706] LustreError: Skipped 1 previous similar message [ 9098.992980] LustreError: 311560:0:(obd_class.h:479:obd_check_dev()) Device 25 not setup [ 9098.995548] LustreError: 311560:0:(obd_class.h:479:obd_check_dev()) Skipped 13 previous similar messages [ 9099.017808] Lustre: server umount lustre-OST0001 complete [ 9102.821293] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 9102.822516] LustreError: 309342:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9102.832289] LustreError: 309342:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 9106.835386] Lustre: server umount lustre-OST0000 complete [ 9113.057075] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 9113.057146] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9113.057746] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9113.057752] Lustre: Skipped 4 previous similar messages [ 9113.068554] Lustre: Skipped 4 previous similar messages [ 9114.377052] LustreError: 311962:0:(obd_class.h:479:obd_check_dev()) Device 28 not setup [ 9114.379881] LustreError: 311962:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 9114.411692] Lustre: server umount lustre-MDT0000 complete [ 9115.980438] LustreError: 307726:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1769082868 with bad export cookie 16270998719830368101 [ 9115.981062] LustreError: MGC192.168.201.141@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9115.985037] LustreError: 307726:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 1 previous similar message [ 9116.144487] Lustre: server umount lustre-MDT0001 complete [ 9120.124528] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing unload_modules_local [ 9121.235251] Key type lgssc unregistered [ 9121.367419] LNet: 312693:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 9121.370223] LNetError: 312693:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 9121.380491] LNet: Removed LNI 192.168.201.141@tcp [ 9121.691563] Key type .llcrypt unregistered [ 9121.692832] Key type ._llcrypt unregistered [ 9131.529113] Key type ._llcrypt registered [ 9131.530482] Key type .llcrypt registered [ 9131.573937] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing load_modules_local [ 9132.002141] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 9132.018464] alg: No test for adler32 (adler32-zlib) [ 9132.879151] Lustre: Lustre: Build Version: 2.17.50_3_g69d656b [ 9132.968300] LNet: Added LNI 192.168.201.141@tcp [8/256/0/180] [ 9134.551122] Key type lgssc registered [ 9134.917312] Lustre: Echo OBD driver; http://www.lustre.org/ [ 9137.469169] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 9137.478078] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9138.643474] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9139.969532] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 9141.805714] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 9144.284306] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 9144.442679] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9145.888969] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 9149.046604] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 9149.168804] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9151.097403] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 9152.813239] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 9154.276485] LustreError: 315621:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9154.286225] LustreError: 315621:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 9155.018408] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 9155.088943] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 9156.922113] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 9158.625303] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid [ 9163.241382] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:33) [ 9163.244667] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:65) [ 9165.479768] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing wait_import_state (FULL|IDLE) os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [ 9165.553926] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 9245.158506] Lustre: lustre-OST0001-osc-MDT0000: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9245.162347] LustreError: lustre-OST0001-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 9245.162761] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 9245.167573] Lustre: Skipped 1 previous similar message [ 9248.497552] LustreError: 317491:0:(obd_class.h:479:obd_check_dev()) Device 27 not setup [ 9248.513875] Lustre: server umount lustre-OST0001 complete [ 9250.276381] LustreError: 315621:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9250.288292] LustreError: 315621:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 9251.809066] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 9251.813977] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9251.821411] Lustre: Skipped 1 previous similar message [ 9251.828877] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9255.395655] LustreError: 315621:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9255.396848] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9255.397191] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9255.398322] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 9255.403950] LustreError: 315621:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 9260.516070] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9260.518494] LustreError: 315621:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9260.580509] Lustre: Skipped 1 previous similar message [ 9260.634514] LustreError: 315621:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 9265.119371] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 9265.147134] LustreError: 317693:0:(obd_class.h:479:obd_check_dev()) Device 23 not setup [ 9265.149836] LustreError: 317693:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 9265.237628] Lustre: server umount lustre-OST0000 complete [ 9270.754165] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9270.762313] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9270.763626] Lustre: Skipped 1 previous similar message [ 9270.772965] Lustre: Skipped 2 previous similar messages [ 9274.128587] LustreError: 317896:0:(obd_class.h:479:obd_check_dev()) Device 10 not setup [ 9274.133420] LustreError: 317896:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 9274.219483] Lustre: server umount lustre-MDT0000 complete [ 9275.879537] LustreError: 314072:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9275.888995] LustreError: 314072:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 9277.076664] LustreError: 314051:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1769083029 with bad export cookie 14255797497110781427 [ 9277.079393] LustreError: MGC192.168.201.141@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9277.083612] LustreError: 314051:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 1 previous similar message [ 9277.191394] LustreError: 318098:0:(obd_class.h:479:obd_check_dev()) Device 18 not setup [ 9277.194707] LustreError: 318098:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 9277.378352] Lustre: server umount lustre-MDT0001 complete [ 9282.561817] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing unload_modules_local [ 9284.091636] Key type lgssc unregistered [ 9284.279643] LNet: 318630:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 9284.286902] LNetError: 318630:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 9284.301714] LNet: Removed LNI 192.168.201.141@tcp [ 9284.784275] Key type .llcrypt unregistered [ 9284.786506] Key type ._llcrypt unregistered [ 9307.952488] Key type ._llcrypt registered [ 9307.954074] Key type .llcrypt registered [ 9308.009654] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing load_modules_local [ 9308.570650] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 9308.596586] alg: No test for adler32 (adler32-zlib) [ 9309.537546] Lustre: Lustre: Build Version: 2.17.50_3_g69d656b [ 9309.666488] LNet: Added LNI 192.168.201.141@tcp [8/256/0/180] [ 9311.263171] Key type lgssc registered [ 9311.791329] Lustre: Echo OBD driver; http://www.lustre.org/ [ 9315.882913] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 9315.893592] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9317.134294] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9319.069374] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 9323.298774] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 9323.480495] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9325.240117] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 9326.516269] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 9329.655380] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 9329.796948] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9332.629783] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 9332.899775] LustreError: 322336:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9332.907053] LustreError: 322336:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 9336.698458] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 9336.805032] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 9339.081743] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 9342.057895] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:65) [ 9342.058375] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:97) [ 9348.270203] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 9356.455461] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 9368.032901] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9368.036363] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9368.037227] Lustre: Skipped 2 previous similar messages [ 9368.040874] Lustre: Skipped 3 previous similar messages [ 9371.918429] LustreError: 324284:0:(obd_class.h:479:obd_check_dev()) Device 10 not setup [ 9371.951135] Lustre: server umount lustre-MDT0000 complete [ 9373.152021] LustreError: 320879:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9373.157695] LustreError: 320879:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 9373.373275] LustreError: 320857:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1769083125 with bad export cookie 16037364152290410157 [ 9373.377730] LustreError: MGC192.168.201.141@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9373.378112] LustreError: 320857:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 9373.452958] LustreError: 324485:0:(obd_class.h:479:obd_check_dev()) Device 18 not setup [ 9373.455324] LustreError: 324485:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 9373.520609] Lustre: server umount lustre-MDT0001 complete [ 9385.464466] LustreError: 324687:0:(obd_class.h:479:obd_check_dev()) Device 23 not setup [ 9385.467076] LustreError: 324687:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 9385.486614] Lustre: server umount lustre-OST0000 complete [ 9397.300867] LustreError: 324889:0:(obd_class.h:479:obd_check_dev()) Device 2 not setup [ 9397.302834] LustreError: 324889:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 9397.351469] Lustre: server umount lustre-OST0001 complete [ 9403.606801] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing load_modules_local [ 9407.912533] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9408.096724] LustreError: 326066:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9408.120510] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9409.610442] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 9413.068820] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 9414.901467] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 9416.106587] Lustre: 327174:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 9418.894661] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 9419.048509] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9419.050304] Lustre: Skipped 1 previous similar message [ 9421.333378] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 9423.143233] LustreError: 328133:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9424.559451] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 9426.870719] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 9433.704784] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:97) [ 9433.704784] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:129) [ 9441.019136] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 9442.568362] Lustre: 329042:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 9448.928283] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 9448.931662] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9448.935641] Lustre: Skipped 1 previous similar message [ 9448.938173] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9454.560327] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9454.562793] Lustre: Skipped 6 previous similar messages [ 9454.854278] LustreError: 329147:0:(obd_class.h:479:obd_check_dev()) Device 10 not setup [ 9454.859796] LustreError: 329147:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 9454.895856] Lustre: server umount lustre-MDT0000 complete [ 9456.385188] LustreError: 326047:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1769083208 with bad export cookie 16037364152290411830 [ 9456.387886] LustreError: MGC192.168.201.141@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9456.389627] LustreError: 326047:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 9456.515315] Lustre: server umount lustre-MDT0001 complete [ 9468.918147] LustreError: 329550:0:(obd_class.h:479:obd_check_dev()) Device 23 not setup [ 9468.920227] LustreError: 329550:0:(obd_class.h:479:obd_check_dev()) Skipped 15 previous similar messages [ 9468.943851] Lustre: server umount lustre-OST0000 complete [ 9480.609258] Lustre: server umount lustre-OST0001 complete [ 9482.028565] Lustre: DEBUG MARKER: SKIP: conf-sanity test_114 skipping SLOW test 114 [ 9482.699029] Lustre: DEBUG MARKER: SKIP: conf-sanity test_115 skipping excluded test 115 [ 9483.415646] Lustre: DEBUG MARKER: == conf-sanity test 116: big size MDT support ============ 07:00:34 (1769083234) [ 9492.706903] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing load_modules_local [ 9498.586639] loop0: detected capacity change from 0 to 1099511627776 [ 9499.575493] SGI XFS with ACLs, security attributes, quota, no debug enabled [ 9499.579216] *** VALIDATE xfs *** [ 9499.584845] XFS (loop0): Mounting V5 Filesystem [ 9499.597457] XFS (loop0): Ending clean mount [ 9500.591907] loop1: detected capacity change from 0 to 18691697672192 [ 9531.520279] LDISKFS-fs (loop1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 9539.702097] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_hostid [ 9544.123881] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing load_modules_local [ 9548.708628] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 9551.713810] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 9554.222276] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 9557.552123] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 9560.739373] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 9560.786819] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9560.902671] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 9560.925196] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 9560.965442] Lustre: lustre-MDT0000: new disk, initializing [ 9561.006980] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9561.011829] Lustre: Skipped 1 previous similar message [ 9561.023509] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 9562.998364] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 9568.636190] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 9568.666114] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 9568.717149] Lustre: 334133:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 9568.745576] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 9568.747951] Lustre: Skipped 1 previous similar message [ 9568.790369] Lustre: lustre-MDT0001: new disk, initializing [ 9568.856137] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 9568.860048] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 9570.748506] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 9574.914622] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 9576.704544] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 9580.451107] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 9580.509115] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 9580.664419] Lustre: lustre-OST0000: new disk, initializing [ 9580.666663] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 9582.244928] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 9582.251911] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 9582.329154] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 9583.743561] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 9587.715345] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 9589.476482] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 9589.568907] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 9591.462479] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 9591.579641] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 9592.801242] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 9592.802246] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9592.805184] LustreError: Skipped 1 previous similar message [ 9592.810015] Lustre: Skipped 3 previous similar messages [ 9592.812786] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9592.816216] Lustre: Skipped 1 previous similar message [ 9597.922154] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9597.924230] Lustre: Skipped 1 previous similar message [ 9598.514375] LustreError: 337071:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 9598.519254] LustreError: 337071:0:(obd_class.h:479:obd_check_dev()) Skipped 9 previous similar messages [ 9598.583646] Lustre: server umount lustre-OST0000 complete [ 9603.041603] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9606.366307] Lustre: server umount lustre-MDT0000 complete [ 9607.879734] LustreError: 334126:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1769083359 with bad export cookie 16037364152290413384 [ 9607.883321] LustreError: MGC192.168.201.141@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9607.884799] LustreError: 334126:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 9608.013105] Lustre: server umount lustre-MDT0001 complete [ 9611.358514] XFS (loop0): Unmounting Filesystem [ 9613.363737] Lustre: DEBUG MARKER: == conf-sanity test 117: lctl get_param return errors properly ========================================================== 07:02:44 (1769083364) [ 9616.241636] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9616.457225] LustreError: 338293:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9616.485422] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9616.489305] Lustre: Skipped 2 previous similar messages [ 9618.094127] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 9621.721326] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 9623.930477] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 9626.340107] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 9627.777807] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 9630.691993] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 9633.390193] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 9635.757645] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 9646.564346] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 9646.564705] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9646.568577] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9646.572062] Lustre: Skipped 2 previous similar messages [ 9646.573597] Lustre: Skipped 2 previous similar messages [ 9647.215139] LustreError: 340636:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 9647.217687] LustreError: 340636:0:(obd_class.h:479:obd_check_dev()) Skipped 17 previous similar messages [ 9647.270261] Lustre: server umount lustre-OST0000 complete [ 9651.227713] Lustre: server umount lustre-MDT0000 complete [ 9651.681826] LustreError: 339006:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9651.690137] LustreError: 339006:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 9653.044255] LustreError: 339912:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1769083405 with bad export cookie 16037364152290414266 [ 9653.046084] LustreError: MGC192.168.201.141@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9653.048443] LustreError: 339912:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 9653.196457] Lustre: server umount lustre-MDT0001 complete [ 9658.373341] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing unload_modules_local [ 9659.616506] Key type lgssc unregistered [ 9659.762520] LNet: 341568:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 9659.765399] LNetError: 341568:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 9659.772490] LNet: Removed LNI 192.168.201.141@tcp [ 9660.104139] Key type .llcrypt unregistered [ 9660.106275] Key type ._llcrypt unregistered [ 9672.385214] Key type ._llcrypt registered [ 9672.386822] Key type .llcrypt registered [ 9672.440464] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing load_modules_local [ 9672.829336] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 9672.881108] alg: No test for adler32 (adler32-zlib) [ 9673.784125] Lustre: Lustre: Build Version: 2.17.50_3_g69d656b [ 9673.879956] LNet: Added LNI 192.168.201.141@tcp [8/256/0/180] [ 9675.465838] Key type lgssc registered [ 9675.903561] Lustre: Echo OBD driver; http://www.lustre.org/ [ 9679.045757] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 9679.056982] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9680.283269] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9681.940018] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 9685.479142] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 9685.649728] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9687.404616] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 9689.818400] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 9691.272778] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 9694.174618] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 9694.347678] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9696.844436] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 9698.961969] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 9714.641240] Lustre: lustre-MDT0001: Not available for connect from 192.168.201.41@tcp (stopping) [ 9716.193436] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9716.198332] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 9716.200302] Lustre: Skipped 1 previous similar message [ 9718.472071] LustreError: 345408:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [ 9718.501986] Lustre: server umount lustre-MDT0001 complete [ 9719.743819] LustreError: 343088:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.201.41@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9719.750590] LustreError: 343088:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 9719.794779] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 9721.313748] LustreError: 343092:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9721.318626] LustreError: 343092:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 9722.846486] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 9722.900917] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [ 9722.903156] Lustre: Found index 1 for lustre-MDT0001, updating log [ 9722.918559] Lustre: 343081:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 9723.040663] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9724.533067] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [ 9728.488639] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 9728.498645] LustreError: 346435:0:(obd_mount.c:188:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [ 9728.506362] LustreError: 346435:0:(tgt_mount.c:669:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [ 9728.512394] LustreError: 343084:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [ 9728.540295] LustreError: 346438:0:(obd_config.c:2001:class_config_llog_handler()) MGC192.168.201.141@tcp: cfg command failed: rc = -17 [ 9728.543764] Lustre: cmd=cf001 0:lustre-MDT0001-osp-MDT0000 1:osp 2:lustre-MDT0000-mdtlov_UUID [ 9728.543764] [10030.564553] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [10030.573075] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10030.577484] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [10030.590398] Lustre: Skipped 2 previous similar messages [10030.610460] Lustre: Skipped 1 previous similar message [10035.684397] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [10035.690252] Lustre: Skipped 2 previous similar messages [10040.810418] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [10040.816452] Lustre: Skipped 2 previous similar messages [10042.335694] Lustre: lustre-MDT0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [10042.443547] LustreError: 355578:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [10042.449382] LustreError: 355578:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [10042.503469] Lustre: server umount lustre-MDT0001 complete [10044.553484] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [10045.921996] LustreError: 343092:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10045.936594] LustreError: 343092:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [10050.442785] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [10050.514908] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [10050.521765] Lustre: Found index 1 for lustre-MDT0001, updating log [10050.550870] Lustre: 343081:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [10050.817857] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10053.393843] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [10056.178521] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [10056.210267] LustreError: 356558:0:(obd_mount.c:188:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [10056.213621] LustreError: 356558:0:(tgt_mount.c:669:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [10056.220681] LustreError: 343084:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [10056.223642] LustreError: 343084:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [10056.257399] LustreError: 356561:0:(obd_config.c:689:class_setup()) Device 8 already setup (type osp) [10056.261437] LustreError: 356561:0:(obd_config.c:797:class_detach()) OBD device 8 still set up [10056.266307] LustreError: 356561:0:(obd_config.c:2001:class_config_llog_handler()) MGC192.168.201.141@tcp: cfg command failed: rc = -17 [10056.272355] Lustre: cmd=cf003 0:lustre-MDT0001-osp-MDT0000 1:lustre-MDT0001_UUID 2:192.168.201.141@tcp [10056.272355] [10358.240362] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [10358.248962] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10358.272410] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [10358.275167] Lustre: Skipped 1 previous similar message [10361.325762] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10362.813799] LustreError: 365597:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [10362.818496] LustreError: 365597:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [10362.925627] Lustre: server umount lustre-MDT0001 complete [10364.796435] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [10364.870162] LustreError: 353832:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.201.41@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [10364.893296] LustreError: 353832:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [10369.998536] LustreError: 344707:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.201.41@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [10370.012238] LustreError: 344707:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [10370.607468] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [10370.671102] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [10370.674648] Lustre: Found index 1 for lustre-MDT0001, updating log [10370.705679] Lustre: 343081:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [10370.931322] Lustre: lustre-MDT0001: Imperative Recovery enabled, recovery window shrunk from 60-180 down to 60-180 [10373.738527] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [10376.177820] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [10380.279210] LustreError: 366674:0:(obd_mount.c:188:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [10380.283126] LustreError: 366674:0:(tgt_mount.c:669:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [10380.286679] LustreError: 343084:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [10380.288455] LustreError: 343084:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [10380.313894] LustreError: 366677:0:(lod_lov.c:212:lod_add_device()) lustre-MDT0001-osp-MDT0000: cannot connect to next dev lustre-MDT0001_UUID (-114) [10380.322161] LustreError: 366677:0:(obd_config.c:2001:class_config_llog_handler()) MGC192.168.201.141@tcp: cfg command failed: rc = -114 [10380.327397] Lustre: cmd=cf014 0:lustre-MDT0000-mdtlov 1:lustre-MDT0001_UUID 2:1 3:1 [10380.327397] [10693.607159] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [10693.607303] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10693.613457] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10693.618367] Lustre: Skipped 1 previous similar message [10693.629552] Lustre: Skipped 5 previous similar messages [10696.533860] LustreError: 376380:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup [10696.537657] LustreError: 376380:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [10696.628093] Lustre: server umount lustre-MDT0000 complete [10698.725897] LustreError: 343088:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10698.745831] LustreError: 343088:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [10698.930737] LustreError: 343073:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1769084451 with bad export cookie 17173920734757328720 [10698.936942] LustreError: MGC192.168.201.141@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10698.938331] LustreError: 343073:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 3 previous similar messages [10703.844414] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10703.851413] Lustre: Skipped 1 previous similar message [10705.111923] LustreError: 376581:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [10705.117283] LustreError: 376581:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [10705.236358] Lustre: server umount lustre-MDT0001 complete [10707.456969] Lustre: server umount lustre-OST0000 complete [10712.860906] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_hostid [10718.713172] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing load_modules_local [10724.314529] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [10728.139457] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [10731.343836] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [10734.839659] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [10738.767907] Lustre: DEBUG MARKER: == conf-sanity test 120: cross-target rename should not create bad symlinks ========================================================== 07:21:29 (1769084489) [10742.394624] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [10742.438142] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10742.582022] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [10742.599937] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [10742.661806] Lustre: lustre-MDT0000: new disk, initializing [10742.727146] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10742.740119] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [10744.786890] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [10750.595716] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [10750.637667] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [10750.685701] Lustre: 379316:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [10750.718263] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [10750.721840] Lustre: Skipped 1 previous similar message [10750.765698] Lustre: lustre-MDT0001: new disk, initializing [10750.826062] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10750.849653] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [10750.854170] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [10752.905727] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [10757.618734] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [10759.930606] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [10763.699898] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [10763.767460] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [10763.892233] Lustre: lustre-OST0000: new disk, initializing [10763.895802] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [10763.945183] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10766.396147] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [10766.400514] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [10766.460406] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [10767.464853] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_default_debug -1 all [10772.275716] Lustre: DEBUG MARKER: oleg141-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [10780.133412] LustreError: lustre-MDT0000-osp-MDT0001: operation out_update to node 0@lo failed: rc = -107 [10780.137507] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10780.143381] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10780.145886] Lustre: Skipped 1 previous similar message [10782.868122] LustreError: 381757:0:(obd_class.h:479:obd_check_dev()) Device 22 not setup [10782.876558] LustreError: 381757:0:(obd_class.h:479:obd_check_dev()) Skipped 11 previous similar messages [10782.925499] Lustre: server umount lustre-MDT0000 complete [10784.992030] LustreError: 380097:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1769084537 with bad export cookie 17173920734757330673 [10785.000215] LustreError: MGC192.168.201.141@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10785.004350] LustreError: 380097:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 2 previous similar messages [10785.004871] LustreError: 380110:0:(osp_object.c:617:osp_attr_get()) lustre-MDT0000-osp-MDT0001: osp_attr_get update error [0x200000401:0x2:0x0]: rc = -5 [10785.193363] Lustre: server umount lustre-MDT0001 complete [10797.540521] Lustre: server umount lustre-OST0000 complete [10813.609144] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing set_hostid [10818.442311] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing load_modules_local [10823.494625] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [10826.820240] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [10830.228147] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [10833.656053] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [10834.550109] Lustre: DEBUG MARKER: == conf-sanity test complete, duration 10521 sec ========= 07:23:05 (1769084585) [10835.574108] Lustre: DEBUG MARKER: === conf-sanity: start cleanup 07:23:06 (1769084586) === [10837.575319] Lustre: DEBUG MARKER: === conf-sanity: finish cleanup 07:23:08 (1769084588) === [10858.064271] Lustre: DEBUG MARKER: oleg141-server.virtnet: executing unload_modules_local [10859.620480] Key type lgssc unregistered [10859.774760] LNet: 387265:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [10859.780587] LNetError: 387265:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [10859.800895] LNet: Removed LNI 192.168.201.141@tcp [10860.335082] Key type .llcrypt unregistered [10860.337748] Key type ._llcrypt unregistered