[ 0.000000] Linux version 4.18.0rh8.10-debug (green@maintenance) (gcc version 8.5.0 20210514 (Red Hat 8.5.0-26) (GCC)) #2 SMP Mon Jul 14 01:24:22 EDT 2025 [ 0.000000] Command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers' [ 0.000000] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256 [ 0.000000] x86/fpu: Enabled xstate features 0x7, context size is 832 bytes, using 'standard' format. [ 0.000000] signal: max sigframe size: 1776 [ 0.000000] BIOS-provided physical RAM map: [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x00000000bffcdfff] usable [ 0.000000] BIOS-e820: [mem 0x00000000bffce000-0x00000000bfffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x0000000146dfffff] usable [ 0.000000] NX (Execute Disable) protection: active [ 0.000000] SMBIOS 2.8 present. [ 0.000000] DMI: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 0.000000] Hypervisor detected: KVM [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00 [ 0.000000] kvm-clock: using sched offset of 473013451 cycles [ 0.000000] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns [ 0.000000] tsc: Detected 2399.998 MHz processor [ 0.000000] last_pfn = 0x146e00 max_arch_pfn = 0x400000000 [ 0.000000] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT [ 0.000000] last_pfn = 0xbffce max_arch_pfn = 0x400000000 [ 0.000000] found SMP MP-table at [mem 0x000f54b0-0x000f54bf] [ 0.000000] RAMDISK: [mem 0xbcc54000-0xbffbffff] [ 0.000000] ACPI: Early table checksum verification disabled [ 0.000000] ACPI: RSDP 0x00000000000F52D0 000014 (v00 BOCHS ) [ 0.000000] ACPI: RSDT 0x00000000BFFE247C 000034 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACP 0x00000000BFFE2318 000074 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: DSDT 0x00000000BFFE0040 0022D8 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACS 0x00000000BFFE0000 000040 [ 0.000000] ACPI: APIC 0x00000000BFFE238C 000090 (v03 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: HPET 0x00000000BFFE241C 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: WAET 0x00000000BFFE2454 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: Reserving FACP table memory at [mem 0xbffe2318-0xbffe238b] [ 0.000000] ACPI: Reserving DSDT table memory at [mem 0xbffe0040-0xbffe2317] [ 0.000000] ACPI: Reserving FACS table memory at [mem 0xbffe0000-0xbffe003f] [ 0.000000] ACPI: Reserving APIC table memory at [mem 0xbffe238c-0xbffe241b] [ 0.000000] ACPI: Reserving HPET table memory at [mem 0xbffe241c-0xbffe2453] [ 0.000000] ACPI: Reserving WAET table memory at [mem 0xbffe2454-0xbffe247b] [ 0.000000] No NUMA configuration found [ 0.000000] Faking a node at [mem 0x0000000000000000-0x0000000146dfffff] [ 0.000000] NODE_DATA(0) allocated [mem 0x1465a3000-0x1465cdfff] [ 0.000000] Reserving 256MB of memory at 2752MB for crashkernel (System RAM: 4205MB) [ 0.000000] Zone ranges: [ 0.000000] DMA [mem 0x0000000000001000-0x0000000000ffffff] [ 0.000000] DMA32 [mem 0x0000000001000000-0x00000000ffffffff] [ 0.000000] Normal [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Device empty [ 0.000000] Movable zone start for each node [ 0.000000] Early memory node ranges [ 0.000000] node 0: [mem 0x0000000000001000-0x000000000009efff] [ 0.000000] node 0: [mem 0x0000000000100000-0x00000000bffcdfff] [ 0.000000] node 0: [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Zeroed struct page in unavailable ranges: 4756 pages [ 0.000000] Initmem setup node 0 [mem 0x0000000000001000-0x0000000146dfffff] [ 0.000000] ACPI: PM-Timer IO Port: 0x608 [ 0.000000] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1]) [ 0.000000] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23 [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level) [ 0.000000] Using ACPI (MADT) for SMP configuration information [ 0.000000] ACPI: HPET id: 0x8086a201 base: 0xfed00000 [ 0.000000] TSC deadline timer available [ 0.000000] smpboot: Allowing 4 CPUs, 0 hotplug CPUs [ 0.000000] kvm-guest: KVM setup pv remote TLB flush [ 0.000000] kvm-guest: setup PV sched yield [ 0.000000] PM: Registered nosave memory: [mem 0x00000000-0x00000fff] [ 0.000000] PM: Registered nosave memory: [mem 0x0009f000-0x0009ffff] [ 0.000000] PM: Registered nosave memory: [mem 0x000a0000-0x000effff] [ 0.000000] PM: Registered nosave memory: [mem 0x000f0000-0x000fffff] [ 0.000000] PM: Registered nosave memory: [mem 0xbffce000-0xbfffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xc0000000-0xfeffbfff] [ 0.000000] PM: Registered nosave memory: [mem 0xfeffc000-0xfeffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xff000000-0xfffbffff] [ 0.000000] PM: Registered nosave memory: [mem 0xfffc0000-0xffffffff] [ 0.000000] [mem 0xc0000000-0xfeffbfff] available for PCI devices [ 0.000000] Booting paravirtualized kernel on KVM [ 0.000000] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns [ 0.000000] setup_percpu: NR_CPUS:8192 nr_cpumask_bits:4 nr_cpu_ids:4 nr_node_ids:1 [ 0.000000] percpu: Embedded 63 pages/cpu s221184 r8192 d28672 u524288 [ 0.000000] kvm-guest: PV spinlocks enabled [ 0.000000] PV qspinlock hash table entries: 256 (order: 0, 4096 bytes, linear) [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 1059606 [ 0.000000] Policy zone: Normal [ 0.000000] Kernel command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] Specific versions of hardware are certified with Red Hat Enterprise Linux 8. Please see the list of hardware certified with Red Hat Enterprise Linux 8 at https://catalog.redhat.com. [ 0.000000] audit: disabled (until reboot) [ 0.000000] software IO TLB: area num 4. [ 0.000000] Memory: 2895240K/4306352K available (18435K kernel code, 11221K rwdata, 7248K rodata, 2908K init, 18040K bss, 524580K reserved, 0K cma-reserved) [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1 [ 0.000000] kmemleak: Kernel memory leak detector disabled [ 0.000000] ftrace: allocating 41240 entries in 162 pages [ 0.000000] ftrace: allocated 162 pages with 3 groups [ 0.000000] rcu: Hierarchical RCU implementation. [ 0.000000] rcu: RCU event tracing is enabled. [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=8192 to nr_cpu_ids=4. [ 0.000000] rcu: RCU callback double-/use-after-free debug enabled. [ 0.000000] Rude variant of Tasks RCU enabled. [ 0.000000] Tracing variant of Tasks RCU enabled. [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies. [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4 [ 0.000000] NR_IRQS: 524544, nr_irqs: 456, preallocated irqs: 16 [ 0.000000] random: get_random_bytes called from start_kernel+0x622/0x9a8 with crng_init=0 [ 0.001000] Console: colour *CGA 80x25 [ 0.001000] printk: console [ttyS1] enabled [ 0.001000] ACPI: Core revision 20220331 [ 0.001000] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns [ 0.001012] APIC: Switch to symmetric I/O mode setup [ 0.002302] x2apic enabled [ 0.003006] Switched APIC routing to physical x2apic. [ 0.004000] kvm-guest: setup PV IPIs [ 0.006000] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1 [ 0.006000] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 0.007016] Calibrating delay loop (skipped) preset value.. 4799.99 BogoMIPS (lpj=2399998) [ 0.008006] pid_max: default: 32768 minimum: 301 [ 0.009112] LSM: Security Framework initializing [ 0.010033] Yama: becoming mindful. [ 0.011024] SELinux: Initializing. [ 0.012047] *** VALIDATE selinux *** [ 0.019343] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, vmalloc) [ 0.023630] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, vmalloc) [ 0.025011] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.026103] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.028014] *** VALIDATE tmpfs *** [ 0.029000] *** VALIDATE proc *** [ 0.029223] *** VALIDATE cgroup *** [ 0.030000] *** VALIDATE cgroup2 *** [ 0.030241] x86/cpu: User Mode Instruction Prevention (UMIP) activated [ 0.031124] Last level iTLB entries: 4KB 0, 2MB 0, 4MB 0 [ 0.032004] Last level dTLB entries: 4KB 0, 2MB 0, 4MB 0, 1GB 0 [ 0.033021] Spectre V2 : User space: Vulnerable [ 0.034005] Speculative Store Bypass: Vulnerable [ 0.036872] debug: unmapping init [mem 0xffffffff96659000-0xffffffff96660fff] [ 0.038231] smpboot: CPU0: Intel(R) Xeon(R) CPU E5-2695 v2 @ 2.40GHz (family: 0x6, model: 0x3e, stepping: 0x4) [ 0.039750] Performance Events: IvyBridge events, full-width counters, Intel PMU driver. [ 0.040024] ... version: 2 [ 0.041008] ... bit width: 48 [ 0.042008] ... generic registers: 4 [ 0.043008] ... value mask: 0000ffffffffffff [ 0.044008] ... max period: 00007fffffffffff [ 0.045009] ... fixed-purpose events: 3 [ 0.046007] ... event mask: 000000070000000f [ 0.047247] rcu: Hierarchical SRCU implementation. [ 0.049414] smp: Bringing up secondary CPUs ... [ 0.050517] x86: Booting SMP configuration: [ 0.051018] .... node #0, CPUs: #1 #2 #3 [ 0.060173] smp: Brought up 1 node, 4 CPUs [ 0.062009] smpboot: Max logical packages: 1 [ 0.063015] smpboot: Total of 4 processors activated (19199.98 BogoMIPS) [ 0.106727] node 0 deferred pages initialised in 42ms [ 0.110484] devtmpfs: initialized [ 0.112378] x86/mm: Memory block size: 128MB [ 0.114088] gcov: version magic: 0x41383552 [ 0.116285] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns [ 0.118075] futex hash table entries: 1024 (order: 4, 65536 bytes, vmalloc) [ 0.120390] pinctrl core: initialized pinctrl subsystem [ 0.121260] [ 0.121545] ************************************************************* [ 0.123010] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.125007] ** ** [ 0.126005] ** IOMMU DebugFS SUPPORT HAS BEEN ENABLED IN THIS KERNEL ** [ 0.127008] ** ** [ 0.129008] ** This means that this kernel is built to expose internal ** [ 0.130006] ** IOMMU data structures, which may compromise security on ** [ 0.132006] ** your system. ** [ 0.133008] ** ** [ 0.135008] ** If you see this message and you are not debugging the ** [ 0.136004] ** kernel, report this immediately to your vendor! ** [ 0.137005] ** ** [ 0.139005] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.140004] ************************************************************* [ 0.141698] NET: Registered protocol family 16 [ 0.144393] DMA: preallocated 512 KiB GFP_KERNEL pool for atomic allocations [ 0.147049] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations [ 0.149044] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations [ 0.153669] cpuidle: using governor menu [ 0.155848] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5 [ 0.158407] PCI: Using configuration type 1 for base access [ 0.160122] core: PMU erratum BJ122, BV98, HSD29 worked around, HT is on [ 0.171168] HugeTLB registered 1.00 GiB page size, pre-allocated 0 pages [ 0.172000] HugeTLB registered 2.00 MiB page size, pre-allocated 0 pages [ 0.174105] cryptd: max_cpu_qlen set to 1000 [ 0.175232] ACPI: Added _OSI(Module Device) [ 0.176000] ACPI: Added _OSI(Processor Device) [ 0.176000] ACPI: Added _OSI(3.0 _SCP Extensions) [ 0.177010] ACPI: Added _OSI(Processor Aggregator Device) [ 0.182732] ACPI: 1 ACPI AML tables successfully acquired and loaded [ 0.189427] ACPI: Interpreter enabled [ 0.191080] ACPI: PM: (supports S0 S3 S4 S5) [ 0.192007] ACPI: Using IOAPIC for interrupt routing [ 0.194141] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug [ 0.198000] ACPI: Enabled 2 GPEs in block 00 to 0F [ 0.211657] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff]) [ 0.225059] acpi PNP0A03:00: _OSC: OS supports [ASPM ClockPM Segments MSI HPX-Type3] [ 0.228016] acpi PNP0A03:00: _OSC: not requesting OS control; OS requires [ExtendedConfig ASPM ClockPM MSI] [ 0.233163] acpi PNP0A03:00: fail to add MMCONFIG information, can't access extended PCI configuration space under this bridge. [ 0.243764] acpiphp: Slot [2] registered [ 0.245070] acpiphp: Slot [5] registered [ 0.246107] acpiphp: Slot [6] registered [ 0.247057] acpiphp: Slot [7] registered [ 0.248088] acpiphp: Slot [8] registered [ 0.249000] acpiphp: Slot [9] registered [ 0.249000] acpiphp: Slot [10] registered [ 0.251117] acpiphp: Slot [3] registered [ 0.253086] acpiphp: Slot [4] registered [ 0.254074] acpiphp: Slot [11] registered [ 0.255074] acpiphp: Slot [12] registered [ 0.256062] acpiphp: Slot [13] registered [ 0.257082] acpiphp: Slot [14] registered [ 0.258121] acpiphp: Slot [15] registered [ 0.259054] acpiphp: Slot [16] registered [ 0.260055] acpiphp: Slot [17] registered [ 0.261053] acpiphp: Slot [18] registered [ 0.262020] acpiphp: Slot [19] registered [ 0.263025] acpiphp: Slot [20] registered [ 0.264022] acpiphp: Slot [21] registered [ 0.265055] acpiphp: Slot [22] registered [ 0.266038] acpiphp: Slot [23] registered [ 0.267053] acpiphp: Slot [24] registered [ 0.268000] acpiphp: Slot [25] registered [ 0.268000] acpiphp: Slot [26] registered [ 0.268000] acpiphp: Slot [27] registered [ 0.268000] acpiphp: Slot [28] registered [ 0.268000] acpiphp: Slot [29] registered [ 0.271073] acpiphp: Slot [30] registered [ 0.272057] acpiphp: Slot [31] registered [ 0.273068] PCI host bridge to bus 0000:00 [ 0.274010] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window] [ 0.277017] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window] [ 0.278011] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window] [ 0.282021] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window] [ 0.284014] pci_bus 0000:00: root bus resource [mem 0xe0000000000-0xe007fffffff window] [ 0.286018] pci_bus 0000:00: root bus resource [bus 00-ff] [ 0.287136] pci 0000:00:00.0: [8086:1237] type 00 class 0x060000 [ 0.288874] pci 0000:00:01.0: [8086:7000] type 00 class 0x060100 [ 0.291083] pci 0000:00:01.1: [8086:7010] type 00 class 0x010180 [ 0.296630] pci 0000:00:01.1: reg 0x20: [io 0xc320-0xc32f] [ 0.302051] pci 0000:00:01.1: legacy IDE quirk: reg 0x10: [io 0x01f0-0x01f7] [ 0.304015] pci 0000:00:01.1: legacy IDE quirk: reg 0x14: [io 0x03f6] [ 0.306013] pci 0000:00:01.1: legacy IDE quirk: reg 0x18: [io 0x0170-0x0177] [ 0.308009] pci 0000:00:01.1: legacy IDE quirk: reg 0x1c: [io 0x0376] [ 0.310913] pci 0000:00:01.3: [8086:7113] type 00 class 0x068000 [ 0.312831] pci 0000:00:01.3: quirk: [io 0x0600-0x063f] claimed by PIIX4 ACPI [ 0.314031] pci 0000:00:01.3: quirk: [io 0x0700-0x070f] claimed by PIIX4 SMB [ 0.316681] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 [ 0.321014] pci 0000:00:02.0: reg 0x10: [io 0xc300-0xc31f] [ 0.331833] pci 0000:00:02.0: reg 0x20: [mem 0xe0000000000-0xe0000003fff 64bit pref] [ 0.337010] pci 0000:00:02.0: reg 0x30: [mem 0xfeb80000-0xfebbffff pref] [ 0.340948] pci 0000:00:05.0: [1af4:1001] type 00 class 0x010000 [ 0.350015] pci 0000:00:05.0: reg 0x10: [io 0xc000-0xc07f] [ 0.358016] pci 0000:00:05.0: reg 0x14: [mem 0xfebc0000-0xfebc0fff] [ 0.372011] pci 0000:00:05.0: reg 0x20: [mem 0xe0000004000-0xe0000007fff 64bit pref] [ 0.384150] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 [ 0.390030] pci 0000:00:06.0: reg 0x10: [io 0xc080-0xc0ff] [ 0.403028] pci 0000:00:06.0: reg 0x14: [mem 0xfebc1000-0xfebc1fff] [ 0.424022] pci 0000:00:06.0: reg 0x20: [mem 0xe0000008000-0xe000000bfff 64bit pref] [ 0.436249] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 [ 0.479050] pci 0000:00:07.0: reg 0x10: [io 0xc100-0xc17f] [ 0.496028] pci 0000:00:07.0: reg 0x14: [mem 0xfebc2000-0xfebc2fff] [ 0.519015] pci 0000:00:07.0: reg 0x20: [mem 0xe000000c000-0xe000000ffff 64bit pref] [ 0.528541] pci 0000:00:08.0: [1af4:1001] type 00 class 0x010000 [ 0.535013] pci 0000:00:08.0: reg 0x10: [io 0xc180-0xc1ff] [ 0.546015] pci 0000:00:08.0: reg 0x14: [mem 0xfebc3000-0xfebc3fff] [ 0.565017] pci 0000:00:08.0: reg 0x20: [mem 0xe0000010000-0xe0000013fff 64bit pref] [ 0.579090] pci 0000:00:09.0: [1af4:1001] type 00 class 0x010000 [ 0.585012] pci 0000:00:09.0: reg 0x10: [io 0xc200-0xc27f] [ 0.591015] pci 0000:00:09.0: reg 0x14: [mem 0xfebc4000-0xfebc4fff] [ 0.604015] pci 0000:00:09.0: reg 0x20: [mem 0xe0000014000-0xe0000017fff 64bit pref] [ 0.615702] pci 0000:00:0a.0: [1af4:1001] type 00 class 0x010000 [ 0.621015] pci 0000:00:0a.0: reg 0x10: [io 0xc280-0xc2ff] [ 0.626015] pci 0000:00:0a.0: reg 0x14: [mem 0xfebc5000-0xfebc5fff] [ 0.660013] pci 0000:00:0a.0: reg 0x20: [mem 0xe0000018000-0xe000001bfff 64bit pref] [ 0.679960] ACPI: PCI: Interrupt link LNKA configured for IRQ 10 [ 0.683112] ACPI: PCI: Interrupt link LNKB configured for IRQ 10 [ 0.688434] ACPI: PCI: Interrupt link LNKC configured for IRQ 11 [ 0.692487] ACPI: PCI: Interrupt link LNKD configured for IRQ 11 [ 0.694332] ACPI: PCI: Interrupt link LNKS configured for IRQ 9 [ 0.700184] iommu: Default domain type: Passthrough [ 0.703403] SCSI subsystem initialized [ 0.706117] ACPI: bus type USB registered [ 0.709130] usbcore: registered new interface driver usbfs [ 0.712071] usbcore: registered new interface driver hub [ 0.716067] usbcore: registered new device driver usb [ 0.719871] pps_core: LinuxPPS API ver. 1 registered [ 0.722010] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti [ 0.725074] PTP clock support registered [ 0.727286] EDAC MC: Ver: 3.0.0 [ 0.729959] PCI: Using ACPI for IRQ routing [ 0.732122] NetLabel: Initializing [ 0.734009] NetLabel: domain hash size = 128 [ 0.736014] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO [ 0.738187] NetLabel: unlabeled traffic allowed by default [ 0.741338] vgaarb: loaded [ 0.743462] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0 [ 0.745015] hpet0: 3 comparators, 64-bit 100.000000 MHz counter [ 0.752322] clocksource: Switched to clocksource kvm-clock [ 0.893818] VFS: Disk quotas dquot_6.6.0 [ 0.895238] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) [ 0.897449] *** VALIDATE ramfs *** [ 0.898568] *** VALIDATE hugetlbfs *** [ 0.900013] pnp: PnP ACPI init [ 0.902216] pnp: PnP ACPI: found 6 devices [ 0.918093] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns [ 0.920686] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window] [ 0.922269] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window] [ 0.924014] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window] [ 0.926032] pci_bus 0000:00: resource 7 [mem 0xc0000000-0xfebfffff window] [ 0.928049] pci_bus 0000:00: resource 8 [mem 0xe0000000000-0xe007fffffff window] [ 0.930474] NET: Registered protocol family 2 [ 0.932545] IP idents hash table entries: 131072 (order: 8, 1048576 bytes, vmalloc) [ 0.937072] tcp_listen_portaddr_hash hash table entries: 4096 (order: 5, 163840 bytes, vmalloc) [ 0.939940] TCP established hash table entries: 65536 (order: 7, 524288 bytes, vmalloc) [ 0.945340] TCP bind hash table entries: 65536 (order: 9, 2097152 bytes, vmalloc) [ 0.948809] TCP: Hash tables configured (established 65536 bind 65536) [ 0.951799] MPTCP token hash table entries: 8192 (order: 6, 393216 bytes, vmalloc) [ 0.955244] UDP hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 0.958457] UDP-Lite hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 0.961196] NET: Registered protocol family 1 [ 0.963515] RPC: Registered named UNIX socket transport module. [ 0.965241] RPC: Registered udp transport module. [ 0.967028] RPC: Registered tcp transport module. [ 0.968690] RPC: Registered tcp NFSv4.1 backchannel transport module. [ 0.970858] NET: Registered protocol family 44 [ 0.972283] pci 0000:00:00.0: Limiting direct PCI/PCI transfers [ 0.974102] pci 0000:00:01.0: PIIX3: Enabling Passive Release [ 0.975978] pci 0000:00:01.0: Activating ISA DMA hang workarounds [ 0.979423] PCI: CLS 0 bytes, default 64 [ 0.982856] Unpacking initramfs... [ 2.893743] debug: unmapping init [mem 0xffff9fc33cc54000-0xffff9fc33ffbffff] [ 2.926408] PCI-DMA: Using software bounce buffering for IO (SWIOTLB) [ 2.930649] software IO TLB: mapped [mem 0x00000000a8000000-0x00000000ac000000] (64MB) [ 2.937666] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 3.910929] Initialise system trusted keyrings [ 3.912650] Key type blacklist registered [ 3.914436] workingset: timestamp_bits=36 max_order=20 bucket_order=0 [ 3.923832] zbud: loaded [ 3.926762] *** VALIDATE nfs *** [ 3.927834] *** VALIDATE nfs4 *** [ 3.929234] pstore: using deflate compression [ 3.933206] Platform Keyring initialized [ 4.261411] NET: Registered protocol family 38 [ 4.266952] Key type asymmetric registered [ 4.269818] Asymmetric key parser 'x509' registered [ 4.274257] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 247) [ 4.280925] io scheduler mq-deadline registered [ 4.284717] io scheduler kyber registered [ 4.289583] io scheduler bfq registered [ 4.295371] atomic64_test: passed for x86-64 platform with CX8 and with SSE [ 4.303929] shpchp: Standard Hot Plug PCI Controller Driver version: 0.4 [ 4.313832] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0 [ 4.324385] ACPI: Power Button [PWRF] [ 4.339464] ACPI: \_SB_.LNKB: Enabled at IRQ 10 [ 4.356984] ACPI: \_SB_.LNKA: Enabled at IRQ 11 [ 4.392561] ACPI: \_SB_.LNKC: Enabled at IRQ 11 [ 4.405482] ACPI: \_SB_.LNKD: Enabled at IRQ 10 [ 4.434842] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled [ 4.476885] 00:03: ttyS1 at I/O 0x2f8 (irq = 3, base_baud = 115200) is a 16550A [ 4.518344] 00:04: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A [ 4.523835] Non-volatile memory driver v1.3 [ 4.526631] Linux agpgart interface v0.103 [ 4.570580] virtio_blk virtio1: [vda] 145872 512-byte logical blocks (74.7 MB/71.2 MiB) [ 4.575225] vda: detected capacity change from 0 to 74686464 [ 4.595260] virtio_blk virtio2: [vdb] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) [ 4.598496] vdb: detected capacity change from 0 to 1073741824 [ 4.616336] virtio_blk virtio3: [vdc] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 4.619814] vdc: detected capacity change from 0 to 2621440000 [ 4.643891] virtio_blk virtio4: [vdd] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 4.648541] vdd: detected capacity change from 0 to 2621440000 [ 4.668722] virtio_blk virtio5: [vde] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 4.671763] vde: detected capacity change from 0 to 4294967296 [ 4.691424] virtio_blk virtio6: [vdf] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 4.694065] vdf: detected capacity change from 0 to 4294967296 [ 4.702213] libphy: Fixed MDIO Bus: probed [ 4.709343] usbcore: registered new interface driver usbserial_generic [ 4.711847] usbserial: USB Serial support registered for generic [ 4.714323] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12 [ 4.718881] serio: i8042 KBD port at 0x60,0x64 irq 1 [ 4.720633] serio: i8042 AUX port at 0x60,0x64 irq 12 [ 4.723201] mousedev: PS/2 mouse device common for all mice [ 4.725901] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input1 [ 4.729976] rtc_cmos 00:05: RTC can wake from S4 [ 4.733204] rtc_cmos 00:05: registered as rtc0 [ 4.736133] rtc_cmos 00:05: alarms up to one day, y3k, 242 bytes nvram, hpet irqs [ 4.738849] intel_pstate: CPU model not supported [ 4.742429] hid: raw HID events driver (C) Jiri Kosina [ 4.744716] usbcore: registered new interface driver usbhid [ 4.745657] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input4 [ 4.750172] usbhid: USB HID core driver [ 4.754914] drop_monitor: Initializing network drop monitor service [ 4.758393] Initializing XFRM netlink socket [ 4.760234] NET: Registered protocol family 10 [ 4.761527] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input3 [ 4.768934] Segment Routing with IPv6 [ 4.772243] NET: Registered protocol family 17 [ 4.774157] mpls_gso: MPLS GSO support [ 4.781321] RAS: Correctable Errors collector initialized. [ 4.783424] AVX version of gcm_enc/dec engaged. [ 4.784841] AES CTR mode by8 optimization enabled [ 4.875886] sched_clock: Marking stable (4875815289, 0)->(5904984913, -1029169624) [ 4.879788] registered taskstats version 1 [ 4.881768] Loading compiled-in X.509 certificates [ 4.884229] zswap: loaded using pool lzo/zbud [ 4.909303] Key type big_key registered [ 4.920913] Key type encrypted registered [ 4.922248] ima: No TPM chip found, activating TPM-bypass! [ 4.924155] ima: Allocated hash algorithm: sha1 [ 4.925811] ima: No architecture policies found [ 4.927627] evm: Initialising EVM extended attributes: [ 4.929516] evm: security.selinux [ 4.930616] evm: security.ima [ 4.931631] evm: security.capability [ 4.932820] evm: HMAC attrs: 0x1 [ 4.935274] rtc_cmos 00:05: setting system clock to 2026-09-02 14:22:46 UTC (1788358966) [ 4.941872] debug: unmapping init [mem 0xffffffff97603000-0xffffffff977fffff] [ 4.944761] debug: unmapping init [mem 0xffffffff96382000-0xffffffff96658fff] [ 4.953181] Write protecting the kernel read-only data: 28672k [ 4.956547] debug: unmapping init [mem 0xffffffff94a03000-0xffffffff94bfffff] [ 4.959255] debug: unmapping init [mem 0xffffffff95314000-0xffffffff953fffff] [ 4.992505] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 5.000469] systemd[1]: Detected virtualization kvm. [ 5.002039] systemd[1]: Detected architecture x86-64. [ 5.003924] systemd[1]: Running in initial RAM disk. Welcome to Rocky Linux 8.10 (Green Obsidian) dracut-049-233.git20240115.el8 (Initramfs)! [ 5.030972] systemd[1]: No hostname configured. [ 5.034406] systemd[1]: Set hostname to . [ 5.038549] random: systemd: uninitialized urandom read (16 bytes read) [ 5.043777] systemd[1]: Initializing machine ID from random generator. [ 5.285271] random: ln: uninitialized urandom read (6 bytes read) [ 6.046768] random: systemd: uninitialized urandom read (16 bytes read) [ 6.054266] systemd[1]: Reached target Initrd Root Device. [ OK ] Reached target Initrd Root Device. [ 6.070199] random: systemd: uninitialized urandom read (16 bytes read) [ 6.087750] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ 6.142893] random: systemd: uninitialized urandom read (16 bytes read) [ 6.145450] systemd[1]: Reached target Paths. [ OK ] Reached target Paths. [ OK ] Listening on Journal Socket (/dev/log). [ OK ] Listening on udev Control Socket. [ OK ] Reached target Slices. [ OK ] Reached target Local Encrypted Volumes. [ OK ] Reached target Local File Systems. [ OK ] Reached target Swap. [ OK ] Reached target Timers. [ OK ] Listening on Journal Socket. Starting Create list of required st…ce nodes for the current kernel... Starting Setup Virtual Console... Starting Journal Service... [ OK ] Started Memstrack Anylazing Service. Starting Apply Kernel Variables... [ OK ] Listening on udev Kernel Socket. [ OK ] Reached target Sockets. Starting Create Volatile Files and Directories... [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Started Setup Virtual Console. [ OK ] Started Apply Kernel Variables. [ OK ] Started Create Volatile Files and Directories. Starting dracut cmdline hook... Starting Create Static Device Nodes in /dev... [ OK ] Started Journal Service. [ OK ] Started Create Static Device Nodes in /dev. [ OK ] Started dracut cmdline hook. Starting dracut pre-udev hook... [ 8.502511] device-mapper: uevent: version 1.0.3 [ 8.510150] device-mapper: ioctl: 4.46.0-ioctl (2022-02-22) initialised: dm-devel@redhat.com [ OK ] Started dracut pre-udev hook. Starting udev Kernel Device Manager... [ OK ] Started udev Kernel Device Manager. Starting dracut pre-trigger hook... [ OK ] Started dracut pre-trigger hook. Starting udev Coldplug all Devices... Mounting Kernel Configuration File System... [ OK ] Mounted Kernel Configuration File System. [ OK ] Started udev Coldplug all Devices. Starting dracut initqueue hook... [ OK ] Reached target System Initialization. [ OK ] Reached target Basic System. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. [ 10.484500] scsi host0: ata_piix [ 10.505950] virtio_net virtio0 ens2: renamed from eth0 [ 10.512561] scsi host1: ata_piix [ 10.515326] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc320 irq 14 [ 10.517711] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc328 irq 15 [ 10.644172] random: fast init done [ 16.467983] random: crng init done [ 16.498734] random: 5 urandom warning(s) missed due to ratelimiting [ 18.714227] hrtimer: interrupt took 9859287 ns [ 19.494765] dracut-initqueue[589]: RTNETLINK answers: File exists Starting nbd nbd0... [ OK ] Started nbd nbd0. [ OK ] Started dracut initqueue hook. Mounting /sysroot... [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. [ 20.646560] EXT4-fs (nbd0): mounted filesystem with ordered data mode. Opts: (null) [ OK ] Mounted /sysroot. [ OK ] Reached target Initrd Root File System. Starting Reload Configuration from the Real Root... [ OK ] Started Reload Configuration from the Real Root. [ OK ] Reached target Initrd File Systems. [ OK ] Reached target Initrd Default Target. Starting dracut pre-pivot and cleanup hook... [ OK ] Started dracut pre-pivot and cleanup hook. Starting Cleaning Up and Shutting Down Daemons... [ OK ] Stopped target Timers. [ OK ] Stopped dracut pre-pivot and cleanup hook. [ OK ] Stopped target Initrd Default Target. [ OK ] Stopped target Initrd Root Device. [ OK ] Stopped target Remote File Systems. [ OK ] Stopped target Remote File Systems (Pre). [ OK ] Stopped dracut initqueue hook. Stopping Hardware RNG Entropy Gatherer Daemon... [ OK ] Stopped Hardware RNG Entropy Gatherer Daemon. [ OK ] Stopped target Basic System. [ OK ] Stopped target Sockets. [ OK ] Stopped target Slices. [ OK ] Stopped target Paths. [ OK ] Stopped target System Initialization. [ OK ] Stopped target Swap. [ OK ] Stopped Apply Kernel Variables. [ OK ] Stopped target Local Encrypted Volumes. [ OK ] Stopped Dispatch Password Requests to Console Directory Watch. [ OK ] Stopped udev Coldplug all Devices. [ OK ] Stopped dracut pre-trigger hook. Stopping udev Kernel Device Manager... [ OK ] Stopped Create Volatile Files and Directories. [ OK ] Stopped target Local File Systems. [ OK ] Stopped udev Kernel Device Manager. [ OK ] Started Cleaning Up and Shutting Down Daemons. [ OK ] Stopped Create Static Device Nodes in /dev. [ OK ] Stopped Create list of required sta…vice nodes for the current kernel. [ OK ] Stopped dracut pre-udev hook. [ OK ] Stopped dracut cmdline hook. [ OK ] Closed udev Kernel Socket. [ OK ] Closed udev Control Socket. Starting Cleanup udevd DB... [ OK ] Started Cleanup udevd DB. [ OK ] Reached target Switch Root. Starting Switch Root... [ 23.397302] printk: systemd: 25 output lines suppressed due to ratelimiting [ 23.991381] SELinux: Disabled at runtime. [ 24.090562] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 24.101373] systemd[1]: Detected virtualization kvm. [ 24.103438] systemd[1]: Detected architecture x86-64. Welcome to Rocky Linux 8.10 (Green Obsidian)! [ 25.594700] systemd[1]: initrd-switch-root.service: Succeeded. [ 25.598848] systemd[1]: Stopped Switch Root. [ OK ] Stopped Switch Root. [ 25.611264] systemd[1]: systemd-journald.service: Service has no hold-off time (RestartSec=0), scheduling restart. [ 25.622313] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. [ 25.630371] systemd[1]: Stopped Journal Service. [ OK ] Stopped Journal Service. [ 25.644023] systemd[1]: Starting Journal Service... Starting Journal Service... [ 25.655791] systemd[1]: Started Forward Password Requests to Wall Directory Watch. [ OK ] Started Forward Password Requests to Wall Directory Watch. [ OK ] Created slice system-getty.slice. Starting Remount Root and Kernel File Systems... [ OK ] Listening on Process Core Dump Socket. Activating swap /dev/disk/by-label/SWAP... [ OK ] Created slice User and Session Slice. Mounting POSIX Message Queue File System... [FAILED] Failed to set up automount Arbitrar…rmats File System Automount Point. [ 26.043154] Adding 1048572k swap on /dev/vdb. Priority:-2 extents:1 across:1048572k FS See 'systemctl status proc-sys-fs-binfmt_misc.automount' for details. Mounting Kernel Debug File System... [ OK ] Stopped target Switch Root. [ OK ] Created slice system-serial\x2dgetty.slice. Mounting Huge Pages File System... [ OK ] Listening on RPCbind Server Activation Socket. [ OK ] Reached target RPC Port Mapper. [ OK ] Stopped target Initrd Root File System. [ OK ] Created slice system-sshd\x2dkeygen.slice. [ OK ] Stopped target Initrd File Systems. [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Reached target Paths. [ OK ] Reached target Local Encrypted Volumes. Starting Apply Kernel Variables... [ OK ] Listening on initctl Compatibility Named Pipe. Starting Create list of required st…ce nodes for the current kernel... [ OK ] Listening on udev Kernel Socket. [ OK ] Reached target Slices. [ OK ] Listening on udev Control Socket. Starting udev Coldplug all Devices... [ OK ] Reached target rpc_pipefs.target. [ OK ] Started Journal Service. [FAILED] Failed to start Remount Root and Kernel File Systems. See 'systemctl status systemd-remount-fs.service' for details. [ OK ] Activated swap /dev/disk/by-label/SWAP. [ OK ] Mounted POSIX Message Queue File System. [ OK ] Mounted Kernel Debug File System. [ OK ] Mounted Huge Pages File System. [ OK ] Started Apply Kernel Variables. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Reached target Swap. Starting Configure read-only root support... Starting Create Static Device Nodes in /dev... Starting Flush Journal to Persistent Storage... [ OK ] Started Flush Journal to Persistent Storage. [ OK ] Started Create Static Device Nodes in /dev. [ OK ] Started udev Coldplug all Devices. Starting udev Kernel Device Manager... [ OK ] Reached target Local File Systems (Pre). Mounting /home/green/git/lustre-release... Mounting /mnt... [ 27.329987] squashfs: version 4.0 (2009/01/31) Phillip Lougher [ OK ] Mounted /home/green/git/lustre-release. [ OK ] Mounted /mnt. [ OK ] Started udev Kernel Device Manager. [ 29.297324] input: PC Speaker as /devices/platform/pcspkr/input/input5 [ 29.766176] piix4_smbus 0000:00:01.3: SMBus Host Controller at 0x700, revision 0 [ 32.889288] RAPL PMU: API unit is 2^-32 Joules, 0 fixed counters, 10737418240 ms ovfl timer [ 33.021324] EDAC sbridge: Ver: 1.1.2 [* ] A start job is running for Configur…-only root support (7s / no limit) [** ] A start job is running for Configur…-only root support (8s / no limit) [*** ] A start job is running for Configur…-only root support (9s / no limit) [ *** ] A start job is running for Configur…-only root support (9s / no limit)[ 35.801753] Key type dns_resolver registered [ *** ] A start job is running for Configur…only root support (10s / no limit)[ 36.363754] NFS: Registering the id_resolver key type [ 36.366134] Key type id_resolver registered [ 36.367853] Key type id_legacy registered [ ***] A start job is running for Configur…only root support (10s / no limit) [ **] A start job is running for Configur…only root support (11s / no limit) [ OK ] Started Configure read-only root support. Starting Load/Save Random Seed... [ OK ] Reached target Local File Systems. Starting Mark the need to relabel after reboot... Starting Rebuild Dynamic Linker Cache... Starting Create Volatile Files and Directories... [ OK ] Started Load/Save Random Seed. [ OK ] Started Mark the need to relabel after reboot. [ OK ] Started Create Volatile Files and Directories. Starting RPC Bind... Starting Update UTMP about System Boot/Shutdown... [ OK ] Started Update UTMP about System Boot/Shutdown. [ OK ] Started RPC Bind. [ OK ] Started Rebuild Dynamic Linker Cache. Starting Update is Completed... [ OK ] Started Update is Completed. [ OK ] Reached target System Initialization. [ OK ] Started dnf makecache --timer. [ OK ] Started daily update of the root trust anchor for DNSSEC. [ OK ] Listening on D-Bus System Message Bus Socket. [ OK ] Reached target Sockets. [ OK ] Started Daily Cleanup of Temporary Directories. [ OK ] Reached target Timers. [ OK ] Reached target Basic System. Starting Login Service... Starting Restore /run/initramfs on shutdown... [ OK ] Started irqbalance daemon. [ OK ] Started D-Bus System Message Bus. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. Starting Network Manager... [ OK ] Reached target sshd-keygen.target. [ OK ] Started Restore /run/initramfs on shutdown. [ OK ] Started Network Manager. Starting Network Manager Wait Online... [ OK ] Reached target Network. Starting GSSAPI Proxy Daemon... Starting OpenSSH server daemon... Starting Dynamic System Tuning Daemon... [ OK ] Started Login Service. [ OK ] Started GSSAPI Proxy Daemon. [ OK ] Reached target NFS client services. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Starting Permit User Sessions... [ OK ] Started OpenSSH server daemon. [ OK ] Started Permit User Sessions. Starting Hostname Service... [ OK ] Started Serial Getty on ttyS1. [ OK ] Started Command Scheduler. [ OK ] Started Serial Getty on ttyS0. [ OK ] Started Getty on tty1. [ OK ] Reached target Login Prompts. [ OK ] Started Hostname Service. Starting Network Manager Script Dispatcher Service... [ OK ] Started Network Manager Script Dispatcher Service. [ OK ] Started Network Manager Wait Online. [ OK ] Reached target Network is Online. Starting System Logging Service... Starting Notify NFS peers of a restart... Starting Crash recovery kernel arming... [ OK ] Started Notify NFS peers of a restart. [ OK ] Started System Logging Service. Starting Authorization Manager... Rocky Linux 8.10 (Green Obsidian) Kernel 4.18.0rh8.10-debug on an x86_64 oleg238-server login: [ 107.975848] libcfs: loading out-of-tree module taints kernel. [ 108.040236] Key type ._llcrypt registered [ 108.042619] Key type .llcrypt registered [ 108.188658] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_hostid [ 137.608538] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing load_modules_local [ 140.347709] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 140.373163] alg: No test for adler32 (adler32-zlib) [ 141.956350] Lustre: Lustre: Build Version: 2.17.56_1_g896c54c [ 143.060387] LNet: Added LNI 192.168.202.138@tcp [8/256/0/180] [ 144.967913] Key type lgssc registered [ 147.380939] Lustre: Echo OBD driver; http://www.lustre.org/ [ 167.218562] ZFS: Loaded module v2.3.2-1, ZFS pool version 5000, ZFS filesystem version 5 [ 203.899366] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing load_modules_local [ 215.903139] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 215.930818] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 217.274101] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 217.312205] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 217.416951] Lustre: lustre-MDT0000: new disk, initializing [ 217.541371] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 217.567235] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 221.741174] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 235.113954] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 235.167568] Lustre: 6506:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 235.195378] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 235.199125] Lustre: Skipped 1 previous similar message [ 235.282608] Lustre: lustre-MDT0001: new disk, initializing [ 235.347310] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 235.367500] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 235.379943] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 239.164121] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 243.974835] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 253.640601] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 253.864214] Lustre: lustre-OST0000: new disk, initializing [ 253.867088] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 253.871560] Lustre: 8447:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 253.932990] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 255.979528] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 255.986512] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 256.033835] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 259.174045] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 273.254954] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 273.436294] Lustre: lustre-OST0001: new disk, initializing [ 273.442337] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 273.451756] Lustre: 9520:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 273.544209] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 279.912876] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 280.101968] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 280.114244] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 280.212658] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 290.806178] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 294.692663] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 301.160544] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing check_logdir /tmp/testlogs/ [ 306.077342] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing yml_node [ 310.528679] Lustre: DEBUG MARKER: Client: 2.17.56.1 [ 313.009161] Lustre: DEBUG MARKER: MDS: 2.17.56.1 [ 315.312936] Lustre: DEBUG MARKER: OSS: 2.17.56.1 [ 316.827035] Lustre: DEBUG MARKER: -----============= acceptance-small: conf-sanity ============----- Wed Sep 2 10:27:56 EDT 2026 [ 333.552129] Lustre: DEBUG MARKER: excepting tests: 102 106 115 32newtarball [ 334.989380] Lustre: DEBUG MARKER: skipping tests SLOW=no: 45 69 106 111 114 [ 351.713595] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 351.715296] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 351.729359] Lustre: Skipped 1 previous similar message [ 351.741341] Lustre: Skipped 3 previous similar messages [ 356.810579] Lustre: server umount lustre-MDT0000 complete [ 359.797314] LustreError: 6501:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788359321 with bad export cookie 828945373121857880 [ 359.797907] LustreError: MGC192.168.202.138@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 359.803955] LustreError: 6501:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 361.952969] LustreError: 6514:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 361.955507] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 361.959736] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 361.974442] LustreError: 6514:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 6 previous similar messages [ 361.994443] Lustre: Skipped 3 previous similar messages [ 364.966363] LustreError: 6515:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 364.971184] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 364.991492] LustreError: 6515:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 365.016354] Lustre: Skipped 2 previous similar messages [ 370.080639] LustreError: 12152:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 370.080959] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 370.086179] LustreError: 12152:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 370.103903] Lustre: Skipped 1 previous similar message [ 374.239936] Lustre: lustre-MDT0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 374.526290] Lustre: server umount lustre-MDT0001 complete [ 378.734975] Lustre: server umount lustre-OST0000 complete [ 383.025038] Lustre: server umount lustre-OST0001 complete [ 390.585898] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_hostid [ 403.225700] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing load_modules_local [ 432.838028] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 432.990493] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 433.019428] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 433.066489] Lustre: lustre-MDT0000: new disk, initializing [ 433.115243] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 433.127523] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 435.808714] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 443.625830] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 443.723660] Lustre: 15074:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 443.787609] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 443.791058] Lustre: Skipped 1 previous similar message [ 443.854388] Lustre: lustre-MDT0001: new disk, initializing [ 443.928307] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 443.967042] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 443.980474] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 446.938202] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 454.556545] Lustre: DEBUG MARKER: oleg238-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 458.921797] Lustre: DEBUG MARKER: oleg238-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 462.998632] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 463.176380] Lustre: lustre-OST0000: new disk, initializing [ 463.178352] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 463.184363] Lustre: 17043:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 463.226577] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 464.673832] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 464.681069] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 464.766186] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 467.205968] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 473.289207] Lustre: DEBUG MARKER: oleg238-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 475.940980] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 476.125694] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 478.680632] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 478.797284] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 480.232567] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 480.239602] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 480.241522] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 480.241764] Lustre: Skipped 1 previous similar message [ 480.254327] LustreError: Skipped 1 previous similar message [ 486.116526] Lustre: server umount lustre-OST0000 complete [ 489.954490] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 489.960456] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 489.969366] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 489.973501] Lustre: Skipped 3 previous similar messages [ 494.728851] Lustre: server umount lustre-MDT0000 complete [ 495.584326] LustreError: 15086:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 495.597332] LustreError: 15086:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 497.008653] LustreError: 15066:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788359458 with bad export cookie 828945373121859406 [ 497.014059] LustreError: MGC192.168.202.138@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 497.018919] LustreError: 15066:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 6 previous similar messages [ 497.231602] Lustre: server umount lustre-MDT0001 complete [ 500.329271] Lustre: DEBUG MARKER: == conf-sanity test 76a: set permanent params with lctl across mounts ========================================================== 10:31:00 (1788359460) [ 503.819555] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 504.081379] LustreError: 19303:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 504.130690] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 506.661878] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 511.692547] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 514.347607] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 519.131657] Lustre: DEBUG MARKER: oleg238-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 522.771598] Lustre: DEBUG MARKER: oleg238-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 527.174326] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 531.280119] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 535.871783] Lustre: DEBUG MARKER: oleg238-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 542.265085] Lustre: Modifying parameter general.osc.*.max_dirty_mb=452 in log params [ 546.786338] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 546.792773] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 546.806365] Lustre: Skipped 3 previous similar messages [ 546.823027] Lustre: Skipped 4 previous similar messages [ 548.768499] Lustre: server umount lustre-MDT0000 complete [ 550.674889] LustreError: 20036:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788359512 with bad export cookie 828945373121860295 [ 550.677867] LustreError: MGC192.168.202.138@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 550.680467] LustreError: 20036:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 551.904837] LustreError: 20227:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 551.912416] LustreError: 20227:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 552.891674] Lustre: server umount lustre-MDT0001 complete [ 555.418957] Lustre: server umount lustre-OST0000 complete [ 568.129142] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing load_modules_local [ 574.539372] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 574.783363] LustreError: 23674:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 574.832686] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 574.845747] Lustre: Skipped 2 previous similar messages [ 577.438238] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 582.502741] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 585.260900] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 586.997504] Lustre: Setting parameter general.debug_raw_pointers=Y in log params [ 590.899461] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 594.817518] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 600.846075] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 600.949741] Lustre: lustre-OST0001: new disk, initializing [ 600.952952] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 600.956504] Lustre: 25987:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 602.587279] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 602.596836] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 602.672326] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 604.728164] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 611.571527] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 613.961546] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 619.309621] Lustre: server umount lustre-MDT0000 complete [ 621.742155] LustreError: 24413:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788359583 with bad export cookie 828945373121861142 [ 621.747645] LustreError: MGC192.168.202.138@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 621.752600] LustreError: 24413:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 623.072841] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 623.077493] Lustre: Skipped 2 previous similar messages [ 623.079186] LustreError: 23671:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 623.079958] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 623.079963] Lustre: Skipped 1 previous similar message [ 623.090604] LustreError: 23671:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 628.050927] Lustre: server umount lustre-MDT0001 complete [ 630.413214] Lustre: server umount lustre-OST0000 complete [ 632.768495] Lustre: server umount lustre-OST0001 complete [ 641.637645] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing load_modules_local [ 648.017520] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 650.444444] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 655.261956] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 657.378877] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 659.140049] Lustre: 30118:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 662.492367] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 665.350514] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 670.069306] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 673.772102] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 678.203891] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 680.256875] Lustre: 31994:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 683.488418] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 683.494013] Lustre: Skipped 3 previous similar messages [ 688.610869] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 688.613159] Lustre: Skipped 10 previous similar messages [ 689.062583] Lustre: server umount lustre-MDT0000 complete [ 690.667103] LustreError: 28948:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788359652 with bad export cookie 828945373121862437 [ 690.670101] LustreError: MGC192.168.202.138@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 690.674292] LustreError: 28948:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 690.817888] Lustre: server umount lustre-MDT0001 complete [ 702.996396] Lustre: server umount lustre-OST0000 complete [ 714.809949] Lustre: server umount lustre-OST0001 complete [ 719.113991] Lustre: DEBUG MARKER: == conf-sanity test 76b: verify params log setup correctly ========================================================== 10:34:39 (1788359679) [ 733.628532] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing load_modules_local [ 738.070157] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 738.266045] LustreError: 34793:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 738.275670] LustreError: 34793:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 5 previous similar messages [ 738.312409] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 738.314455] Lustre: Skipped 7 previous similar messages [ 740.106236] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 743.784025] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 746.002551] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 747.502637] Lustre: 35940:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 750.935956] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 753.966721] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 758.209632] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 761.535808] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 765.764178] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 767.734484] Lustre: 37816:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 773.090446] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 773.095891] Lustre: Skipped 4 previous similar messages [ 776.222346] Lustre: server umount lustre-MDT0000 complete [ 778.073903] LustreError: 37818:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788359739 with bad export cookie 828945373121863536 [ 778.076270] LustreError: MGC192.168.202.138@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 778.079156] LustreError: 37818:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 778.238395] Lustre: server umount lustre-MDT0001 complete [ 790.475595] Lustre: server umount lustre-OST0000 complete [ 801.595322] Lustre: server umount lustre-OST0001 complete [ 805.937519] Lustre: DEBUG MARKER: == conf-sanity test 76c: verify changelog_mask is applied with lctl set_param -P ========================================================== 10:36:06 (1788359766) [ 813.824735] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing load_modules_local [ 818.500742] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 820.524925] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 824.703176] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 826.855168] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 828.422025] Lustre: 41102:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 831.639126] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 835.260227] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 839.636919] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 842.162946] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 851.694423] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 854.064504] Lustre: Setting parameter general.mdd.*.changelog_mask=-CLOSE in log params [ 854.067936] Lustre: Skipped 1 previous similar message [ 859.057321] Lustre: lustre-MDT0000: Not available for connect from 192.168.202.38@tcp (stopping) [ 859.060676] Lustre: Skipped 4 previous similar messages [ 859.617261] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 859.622033] Lustre: Skipped 3 previous similar messages [ 861.858925] Lustre: server umount lustre-MDT0000 complete [ 863.473793] LustreError: 39934:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788359825 with bad export cookie 828945373121864628 [ 863.479293] LustreError: 39934:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 863.479747] LustreError: MGC192.168.202.138@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 868.768174] LustreError: 39949:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 868.774101] LustreError: 39949:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 9 previous similar messages [ 869.683335] Lustre: server umount lustre-MDT0001 complete [ 873.362176] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 873.887827] LustreError: 3641:0:(client.c:1391:ptlrpc_import_delay_req()) @@@ invalidate in flight req@ffff9fc288c60e00 x1875230436170624/t0(0) o250->MGC192.168.202.138@tcp@0@lo:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl Rpc:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 875.929854] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 880.062633] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 882.064747] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 885.282657] Lustre: DEBUG MARKER: oleg238-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 891.190494] Lustre: DEBUG MARKER: mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 5 sec [ 893.598665] Lustre: DEBUG MARKER: oleg238-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 893.923576] Lustre: lustre-MDT0001-lwp-OST0001: Connection restored to 0@lo (at 0@lo) [ 894.464279] Lustre: DEBUG MARKER: mdc.lustre-MDT0001-mdc-*.mds_server_uuid in FULL state after 0 sec [ 903.034906] Lustre: server umount lustre-MDT0000 complete [ 905.038532] LustreError: 40703:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788359866 with bad export cookie 828945373121865727 [ 905.044620] LustreError: MGC192.168.202.138@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 905.046531] LustreError: 40703:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 905.238626] Lustre: server umount lustre-MDT0001 complete [ 916.493661] Lustre: server umount lustre-OST0000 complete [ 928.297956] Lustre: server umount lustre-OST0001 complete [ 932.179533] Lustre: DEBUG MARKER: == conf-sanity test 76d: verify llite.*.xattr_cache can be set by 'lctl set_param -P' correctly ========================================================== 10:38:12 (1788359892) [ 939.440024] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing load_modules_local [ 943.450890] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 945.244866] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 948.696507] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 950.726972] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 951.928994] Lustre: 48732:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 951.932827] Lustre: 48732:0:(mgs_llog.c:1450:mgs_modify_param()) Skipped 1 previous similar message [ 954.644404] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 957.244289] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 961.032631] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 963.486051] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 972.614207] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 974.986947] Lustre: Setting parameter general.llite.*.xattr_cache=0 in log params [ 987.399712] Lustre: server umount lustre-MDT0000 complete [ 989.069101] LustreError: 48334:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788359950 with bad export cookie 828945373121866539 [ 989.070764] LustreError: MGC192.168.202.138@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 989.073463] LustreError: 48334:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 989.152847] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 989.163605] Lustre: Skipped 8 previous similar messages [ 989.260227] Lustre: server umount lustre-MDT0001 complete [ 997.191539] Lustre: server umount lustre-OST0000 complete [ 998.815485] Lustre: server umount lustre-OST0001 complete [ 1003.699493] Lustre: DEBUG MARKER: == conf-sanity test 77: comma-separated MGS NIDs and failover node NIDs ========================================================== 10:39:23 (1788359963) [ 1005.128586] Lustre: DEBUG MARKER: SKIP: conf-sanity test_77 mixed loopback and real device not working [ 1006.223269] Lustre: DEBUG MARKER: == conf-sanity test 78: run resize2fs on MDT and OST filesystems ========================================================== 10:39:26 (1788359966) [ 1011.843649] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_hostid [ 1016.269207] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing load_modules_local [ 1027.932824] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1028.041263] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1028.079438] Lustre: lustre-MDT0000: new disk, initializing [ 1028.118119] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1028.121090] Lustre: Skipped 13 previous similar messages [ 1028.129021] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1030.097886] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 1034.773526] Lustre: DEBUG MARKER: oleg238-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1037.940970] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1038.062504] Lustre: 54803:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1039.584596] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 1039.608283] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 1040.942920] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 1045.681164] Lustre: DEBUG MARKER: oleg238-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1046.699568] Lustre: DEBUG MARKER: create test files [ 1065.444264] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1066.528563] Lustre: server umount lustre-OST0000 complete [ 1068.448509] Lustre: server umount lustre-MDT0000 complete [ 1073.174424] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing unload_modules_local [ 1074.531749] Key type lgssc unregistered [ 1074.690829] LNet: 56427:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1074.695675] LNetError: 56427:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1074.704603] LNet: Removed LNI 192.168.202.138@tcp [ 1075.164305] Key type .llcrypt unregistered [ 1075.167264] Key type ._llcrypt unregistered [ 1088.649549] Key type ._llcrypt registered [ 1088.651145] Key type .llcrypt registered [ 1088.710318] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing load_modules_local [ 1089.138700] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1089.192360] alg: No test for adler32 (adler32-zlib) [ 1090.085731] Lustre: Lustre: Build Version: 2.17.56_1_g896c54c [ 1090.192373] LNet: Added LNI 192.168.202.138@tcp [8/256/0/180] [ 1091.783185] Key type lgssc registered [ 1092.236354] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1095.505409] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1095.522751] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1096.718922] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1098.302557] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 1101.233435] Lustre: DEBUG MARKER: oleg238-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1104.076171] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1104.229050] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1106.830075] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 1107.305188] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:102 to 0x240000400:129) [ 1107.352758] Lustre: 59136:0:(osd_internal.h:1461:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 510 < left 544, rollback = 9 [ 1107.355865] Lustre: 59136:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 32/128/5, destroy: 0/0/0 [ 1107.358570] Lustre: 59136:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 1107.361470] Lustre: 59136:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/18/1, punch: 0/0/0, quota 1/3/0 [ 1107.365179] Lustre: 59136:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 1107.369487] Lustre: 59136:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 1110.144885] Lustre: DEBUG MARKER: oleg238-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1113.881516] Lustre: DEBUG MARKER: check files after expanding the MDT and OST filesystems [ 1116.834488] Lustre: DEBUG MARKER: create more files after expanding the MDT and OST filesystems [ 1122.784842] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1122.787633] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1122.794163] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1125.812423] Lustre: server umount lustre-OST0000 complete [ 1127.417992] Lustre: server umount lustre-MDT0000 complete [ 1131.577629] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing unload_modules_local [ 1132.752345] Key type lgssc unregistered [ 1132.885978] LNet: 60745:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1132.889455] LNetError: 60745:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1132.901692] LNet: Removed LNI 192.168.202.138@tcp [ 1133.270294] Key type .llcrypt unregistered [ 1133.271688] Key type ._llcrypt unregistered [ 1147.066418] Key type ._llcrypt registered [ 1147.069097] Key type .llcrypt registered [ 1147.112123] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing load_modules_local [ 1147.616845] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1147.640066] alg: No test for adler32 (adler32-zlib) [ 1148.531263] Lustre: Lustre: Build Version: 2.17.56_1_g896c54c [ 1148.667768] LNet: Added LNI 192.168.202.138@tcp [8/256/0/180] [ 1150.263495] Key type lgssc registered [ 1150.792730] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1154.425799] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1154.449575] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1155.672519] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1157.346022] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 1160.486929] Lustre: DEBUG MARKER: oleg238-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1163.621035] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1163.758426] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1165.797955] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:140 to 0x240000400:161) [ 1165.828739] Lustre: 63553:0:(osd_internal.h:1461:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 509 < left 544, rollback = 9 [ 1165.832055] Lustre: 63553:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 32/128/6, destroy: 0/0/0 [ 1165.835879] Lustre: 63553:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 1165.838339] Lustre: 63553:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/18/1, punch: 0/0/0, quota 1/3/0 [ 1165.841525] Lustre: 63553:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 1165.843977] Lustre: 63553:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 1166.240908] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 1169.411705] Lustre: DEBUG MARKER: oleg238-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1172.742715] Lustre: DEBUG MARKER: check files after shrinking the MDT and OST filesystems [ 1206.753507] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1206.753937] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1206.766718] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1211.873260] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1212.821402] Lustre: server umount lustre-OST0000 complete [ 1214.788995] Lustre: server umount lustre-MDT0000 complete [ 1219.448896] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing unload_modules_local [ 1220.851840] Key type lgssc unregistered [ 1221.034464] LNet: 65135:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1221.036995] LNetError: 65135:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1221.048184] LNet: Removed LNI 192.168.202.138@tcp [ 1221.443749] Key type .llcrypt unregistered [ 1221.445362] Key type ._llcrypt unregistered [ 1230.453509] Key type ._llcrypt registered [ 1230.454931] Key type .llcrypt registered [ 1230.515230] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_hostid [ 1238.578837] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing load_modules_local [ 1239.152718] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1239.164670] alg: No test for adler32 (adler32-zlib) [ 1240.061765] Lustre: Lustre: Build Version: 2.17.56_1_g896c54c [ 1240.168858] LNet: Added LNI 192.168.202.138@tcp [8/256/0/180] [ 1241.784586] Key type lgssc registered [ 1242.382211] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1261.864779] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1261.877786] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1263.039743] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 1263.059208] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1263.097171] Lustre: lustre-MDT0000: new disk, initializing [ 1263.132707] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1263.144562] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1265.150464] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 1270.474335] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1270.521705] Lustre: 68262:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 1270.550430] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 1270.553924] Lustre: Skipped 1 previous similar message [ 1270.595779] Lustre: lustre-MDT0001: new disk, initializing [ 1270.635926] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 1270.653269] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 1270.657907] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 1272.264588] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 1276.344360] Lustre: DEBUG MARKER: oleg238-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1278.598836] Lustre: DEBUG MARKER: oleg238-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 1281.382486] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1281.507230] Lustre: lustre-OST0000: new disk, initializing [ 1281.508848] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1281.511722] Lustre: 70230:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1281.550210] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1284.078195] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 1288.729070] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 1288.734952] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 1288.780414] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 1288.863706] Lustre: DEBUG MARKER: oleg238-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1290.728985] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 1290.819784] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 1292.643103] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 1292.736323] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 1293.793490] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1293.798683] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1293.801214] Lustre: Skipped 1 previous similar message [ 1293.808472] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1295.733658] Lustre: server umount lustre-OST0000 complete [ 1298.912382] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1298.923159] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1298.926062] Lustre: Skipped 1 previous similar message [ 1303.859824] Lustre: server umount lustre-MDT0000 complete [ 1305.595017] LustreError: 68256:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788360267 with bad export cookie 4128200608092789907 [ 1305.597438] LustreError: MGC192.168.202.138@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1305.599066] LustreError: 68256:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 1305.752235] Lustre: server umount lustre-MDT0001 complete [ 1309.275420] Lustre: DEBUG MARKER: == conf-sanity test 79: format MDT/OST without mgs option (should return errors) ========================================================== 10:44:29 (1788360269) [ 1313.714862] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing load_modules_local [ 1330.967157] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_hostid [ 1334.854159] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing load_modules_local [ 1352.560534] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1352.654756] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 1352.668674] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1352.708971] Lustre: lustre-MDT0000: new disk, initializing [ 1352.739923] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1352.751058] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1354.538566] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 1359.872840] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1359.925905] Lustre: 76241:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 1359.962394] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 1359.966245] Lustre: Skipped 1 previous similar message [ 1360.013263] Lustre: lustre-MDT0001: new disk, initializing [ 1360.050995] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 1360.069853] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 1360.074593] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 1361.785146] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 1366.756827] Lustre: DEBUG MARKER: oleg238-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1369.640626] Lustre: DEBUG MARKER: oleg238-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 1373.320346] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1373.494478] Lustre: lustre-OST0000: new disk, initializing [ 1373.497620] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1373.502875] Lustre: 78209:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1373.564679] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1376.647615] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 1380.356038] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 1380.360597] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 1380.421511] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 1381.270540] Lustre: DEBUG MARKER: oleg238-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1383.005324] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 1383.091537] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 1384.875083] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 1384.964389] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 1390.562571] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 1390.563287] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1390.566446] LustreError: Skipped 1 previous similar message [ 1390.570983] Lustre: Skipped 1 previous similar message [ 1390.574391] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1390.579236] Lustre: Skipped 1 previous similar message [ 1395.684240] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1395.686575] Lustre: Skipped 1 previous similar message [ 1400.287116] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 1400.360732] Lustre: server umount lustre-OST0000 complete [ 1405.920173] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1405.923795] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1405.933439] Lustre: Skipped 1 previous similar message [ 1408.203766] Lustre: server umount lustre-MDT0000 complete [ 1410.283448] LustreError: 77035:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788360371 with bad export cookie 4128200608092790775 [ 1410.284458] LustreError: MGC192.168.202.138@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1410.288899] LustreError: 77035:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1410.490336] Lustre: server umount lustre-MDT0001 complete [ 1414.597268] Lustre: DEBUG MARKER: == conf-sanity test 80: mgc import reconnect race ======== 10:46:14 (1788360374) [ 1417.777151] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1417.976578] LustreError: 80618:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1417.996060] LustreError: 80618:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [ 1418.027795] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1420.084435] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 1423.330862] LustreError: 80618:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1423.338399] LustreError: 80618:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 1424.170346] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1426.305712] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 1430.230443] Lustre: DEBUG MARKER: oleg238-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1433.288147] Lustre: DEBUG MARKER: oleg238-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 1437.140441] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1439.965756] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 1443.300590] Lustre: DEBUG MARKER: oleg238-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1444.949622] Lustre: 83286:0:(genops.c:1795:obd_export_evict_by_uuid()) MGS: evicting fd014609-836c-4763-a088-c1add9443e1c at adminstrative request [ 1448.613819] LustreError: MGC192.168.202.138@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1448.620230] Lustre: *** cfs_fail_loc=906, val=2147483648*** [ 1459.172310] Lustre: MGS: Client fd014609-836c-4763-a088-c1add9443e1c (at 0@lo) reconnecting [ 1459.175846] Lustre: Evicted from MGS (at 0@lo) after server handle changed from 0x394a5094b6ec2369 to 0x394a5094b6ec25fb [ 1459.184103] Lustre: MGC192.168.202.138@tcp: Connection restored to 0@lo (at 0@lo) [ 1478.193849] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1478.271421] Lustre: lustre-OST0001: new disk, initializing [ 1478.273843] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 1478.277552] Lustre: 83677:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1478.317827] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 1478.320428] Lustre: Skipped 2 previous similar messages [ 1480.859705] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 1485.538506] Lustre: DEBUG MARKER: oleg238-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1486.884638] Lustre: server umount lustre-OST0001 complete [ 1487.879423] LustreError: 82440:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1489.888882] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1489.889268] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1489.891903] LustreError: Skipped 1 previous similar message [ 1489.892452] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1489.892456] Lustre: Skipped 1 previous similar message [ 1489.904176] Lustre: Skipped 1 previous similar message [ 1492.963555] LustreError: 82440:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1492.969338] LustreError: 82440:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 1494.621715] Lustre: server umount lustre-OST0000 complete [ 1498.081170] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1498.086703] Lustre: Skipped 1 previous similar message [ 1502.515474] Lustre: server umount lustre-MDT0000 complete [ 1503.200414] LustreError: 80619:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1503.208134] LustreError: 80619:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 1504.217144] LustreError: 80601:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788360465 with bad export cookie 4128200608092792315 [ 1504.219480] LustreError: MGC192.168.202.138@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1504.228258] LustreError: 80601:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 1504.404502] Lustre: server umount lustre-MDT0001 complete [ 1508.408722] Lustre: DEBUG MARKER: == conf-sanity test 81: sparse OST indexing ============== 10:47:48 (1788360468) [ 1509.226196] Lustre: DEBUG MARKER: SKIP: conf-sanity test_81 needs >= 3 OSTs [ 1510.146987] Lustre: DEBUG MARKER: == conf-sanity test 82a: specify OSTs for file (succeed) or directory (succeed) ========================================================== 10:47:50 (1788360470) [ 1510.973361] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82a needs >= 3 OSTs [ 1511.857397] Lustre: DEBUG MARKER: == conf-sanity test 82b: specify OSTs for file with --pool and --ost-list options ========================================================== 10:47:52 (1788360472) [ 1512.635296] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82b needs >= 4 OSTs [ 1513.577166] Lustre: DEBUG MARKER: == conf-sanity test 82c: specify sparse OSTs for setstripe (should not crash) ========================================================== 10:47:53 (1788360473) [ 1514.386496] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82c needs >= 4 OSTs [ 1515.264088] Lustre: DEBUG MARKER: == conf-sanity test 83: ENOSPACE on OST doesn't cause message VFS: Busy inodes after unmount ... ========================================================== 10:47:55 (1788360475) [ 1515.919217] Lustre: DEBUG MARKER: mount the OST /dev/mapper/ost1_flakey as a ldiskfs filesystem [ 1518.417533] loop: module loaded [ 1518.423358] loop0: detected capacity change from 0 to 4294967296 [ 1518.443844] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 1519.079411] Lustre: DEBUG MARKER: run llverfs in partial mode on the OST ldiskfs /mnt/lustre-ost1 [ 1520.528924] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing run_llverfs /mnt/lustre-ost1 -vpl no [ 1522.267273] Lustre: DEBUG MARKER: unmount the OST /dev/mapper/ost1_flakey [ 1534.345160] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1534.363464] LustreError: 87791:0:(obd_config.c:845:class_setup()) setup lustre-OST0000-osd failed (-28) [ 1534.366857] LustreError: 87791:0:(obd_mount.c:259:lustre_start_simple()) lustre-OST0000-osd setup error -28 [ 1534.370825] LustreError: 87791:0:(tgt_mount.c:2579:server_fill_super()) Unable to start osd on /dev/mapper/ost1_flakey: -28 [ 1534.375025] LustreError: 87791:0:(super25.c:178:lustre_fill_super()) llite: Unable to mount : rc = -28 [ 1542.389634] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_hostid [ 1546.242947] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing load_modules_local [ 1560.725473] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1560.827830] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 1560.840133] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1560.876497] Lustre: lustre-MDT0000: new disk, initializing [ 1560.905101] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1560.912665] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1562.415634] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 1567.118344] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1567.164877] Lustre: 90652:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 1567.296516] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 1568.973593] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 1573.473223] Lustre: DEBUG MARKER: oleg238-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1575.808407] Lustre: DEBUG MARKER: oleg238-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 1578.679694] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1578.801772] Lustre: 92619:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1581.302440] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 1584.130236] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 1584.185235] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 1585.899868] Lustre: DEBUG MARKER: oleg238-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1587.659681] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 1587.760384] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 1589.532333] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 1589.637281] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 1594.339083] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 1594.339729] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1594.343481] LustreError: Skipped 1 previous similar message [ 1594.348545] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1594.351839] Lustre: Skipped 6 previous similar messages [ 1596.502307] Lustre: server umount lustre-OST0000 complete [ 1612.255120] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 1612.350047] Lustre: server umount lustre-MDT0000 complete [ 1613.991441] LustreError: 90643:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788360575 with bad export cookie 4128200608092792917 [ 1613.995278] LustreError: MGC192.168.202.138@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1613.997377] LustreError: 90643:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1614.140241] Lustre: server umount lustre-MDT0001 complete [ 1617.778088] Lustre: DEBUG MARKER: == conf-sanity test 84: check recovery_hard_time ========= 10:49:38 (1788360578) [ 1620.799294] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1621.000735] LustreError: 95028:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1623.278306] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 1628.753955] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1632.617586] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 1640.361281] Lustre: DEBUG MARKER: oleg238-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1645.715361] Lustre: DEBUG MARKER: oleg238-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 1651.566487] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1658.099731] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 1664.507756] Lustre: DEBUG MARKER: oleg238-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1669.466945] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 1669.659108] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 1673.971893] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 1674.180489] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 1680.249950] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1680.396614] Lustre: lustre-OST0001: new disk, initializing [ 1680.399035] Lustre: Skipped 2 previous similar messages [ 1680.406288] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 1680.409515] Lustre: Skipped 3 previous similar messages [ 1680.415396] Lustre: 98398:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1685.465543] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 1686.054387] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 1686.062141] Lustre: Skipped 2 previous similar messages [ 1686.069907] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 1686.173828] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 1693.387041] Lustre: DEBUG MARKER: oleg238-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1697.827601] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid 50 [ 1698.027336] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 1702.388414] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [ 1702.588824] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 1718.493216] Lustre: DEBUG MARKER: mds1 REPLAY BARRIER on lustre-MDT0000 [ 1735.130219] Lustre: Failing over lustre-MDT0000 [ 1735.451244] Lustre: server umount lustre-MDT0000 complete [ 1736.640620] LustreError: 95764:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 192.168.202.38@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1736.649188] LustreError: 95764:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [ 1736.673287] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1736.678241] LustreError: Skipped 1 previous similar message [ 1736.679764] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1736.684935] Lustre: Skipped 3 previous similar messages [ 1752.543751] Lustre: 66489:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1788360698/real 1788360698] req@00000000036a4559 x1875231586343808/t0(0) o400->MGC192.168.202.138@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1788360714 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 1752.566167] LustreError: MGC192.168.202.138@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1754.603061] LDISKFS-fs (dm-0): 4 truncates cleaned up [ 1754.605144] LDISKFS-fs (dm-0): recovery complete [ 1754.611655] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1762.785717] LustreError: 66487:0:(client.c:1391:ptlrpc_import_delay_req()) @@@ invalidate in flight req@00000000d4a6a521 x1875231586350720/t0(0) o250->MGC192.168.202.138@tcp@0@lo:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl Rpc:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 1762.938885] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-60 [ 1762.942682] Lustre: Skipped 6 previous similar messages [ 1762.986076] Lustre: lustre-MDT0000: in recovery but waiting for the first client to connect [ 1765.404309] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 1767.349782] Lustre: lustre-MDT0000: Will be in recovery for at least 1:00, or until 3 clients reconnect [ 1767.350682] Lustre: 99798:0:(ldlm_lib.c:2084:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 1768.424509] Lustre: 95028:0:(ldlm_lib.c:2084:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 1768.436382] Lustre: 95028:0:(ldlm_lib.c:2084:extend_recovery_timer()) Skipped 1 previous similar message [ 1768.455592] Lustre: lustre-MDT0000-lwp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 1768.520634] LustreError: 101251:0:(ldlm_lib.c:2697:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 1768.839136] LustreError: 101251:0:(ldlm_lib.c:2697:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 1769.162179] LustreError: 101251:0:(ldlm_lib.c:2697:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 1769.168699] LustreError: 101251:0:(ldlm_lib.c:2697:replay_request_or_update()) Skipped 1 previous similar message [ 1769.487093] LustreError: 101251:0:(ldlm_lib.c:2697:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 1769.494339] LustreError: 101251:0:(ldlm_lib.c:2697:replay_request_or_update()) Skipped 1 previous similar message [ 1769.498983] Lustre: 101251:0:(ldlm_lib.c:2084:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 1769.504915] Lustre: 101251:0:(ldlm_lib.c:2084:extend_recovery_timer()) Skipped 7 previous similar messages [ 1770.458162] LustreError: 101251:0:(ldlm_lib.c:2697:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 1770.466539] LustreError: 101251:0:(ldlm_lib.c:2697:replay_request_or_update()) Skipped 3 previous similar messages [ 1770.791160] LustreError: 101251:0:(ldlm_lib.c:2697:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 1770.799908] LustreError: 101251:0:(ldlm_lib.c:2697:replay_request_or_update()) Skipped 3 previous similar messages [ 1771.757860] Lustre: 101251:0:(ldlm_lib.c:2084:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 1771.775130] Lustre: 101251:0:(ldlm_lib.c:2084:extend_recovery_timer()) Skipped 6 previous similar messages [ 1772.720709] LustreError: 101251:0:(ldlm_lib.c:2697:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 1772.730608] LustreError: 101251:0:(ldlm_lib.c:2697:replay_request_or_update()) Skipped 6 previous similar messages [ 1773.047283] LustreError: 101251:0:(ldlm_lib.c:2697:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 1773.058263] LustreError: 101251:0:(ldlm_lib.c:2697:replay_request_or_update()) Skipped 6 previous similar messages [ 1775.897366] Lustre: 101251:0:(ldlm_lib.c:2084:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 1775.902837] Lustre: 101251:0:(ldlm_lib.c:2084:extend_recovery_timer()) Skipped 12 previous similar messages [ 1776.865635] LustreError: 101251:0:(ldlm_lib.c:2697:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 1776.885744] LustreError: 101251:0:(ldlm_lib.c:2697:replay_request_or_update()) Skipped 12 previous similar messages [ 1777.215185] LustreError: 101251:0:(ldlm_lib.c:2697:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 1777.219693] LustreError: 101251:0:(ldlm_lib.c:2697:replay_request_or_update()) Skipped 12 previous similar messages [ 1784.135804] Lustre: 101251:0:(ldlm_lib.c:2084:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 1784.140842] Lustre: 101251:0:(ldlm_lib.c:2084:extend_recovery_timer()) Skipped 25 previous similar messages [ 1785.091459] LustreError: 101251:0:(ldlm_lib.c:2697:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 1785.101379] LustreError: 101251:0:(ldlm_lib.c:2697:replay_request_or_update()) Skipped 25 previous similar messages [ 1785.424176] LustreError: 101251:0:(ldlm_lib.c:2697:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 1785.429565] LustreError: 101251:0:(ldlm_lib.c:2697:replay_request_or_update()) Skipped 25 previous similar messages [ 1800.225954] Lustre: 101251:0:(ldlm_lib.c:2084:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 1800.232097] Lustre: 101251:0:(ldlm_lib.c:2084:extend_recovery_timer()) Skipped 50 previous similar messages [ 1801.184098] LustreError: 101251:0:(ldlm_lib.c:2697:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 1801.190249] LustreError: 101251:0:(ldlm_lib.c:2697:replay_request_or_update()) Skipped 50 previous similar messages [ 1801.519136] LustreError: 101251:0:(ldlm_lib.c:2697:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 1801.524452] LustreError: 101251:0:(ldlm_lib.c:2697:replay_request_or_update()) Skipped 50 previous similar messages [ 1827.103129] Lustre: 101251:0:(ldlm_lib.c:2393:target_recovery_overseer()) lustre-MDT0000 recovery is aborted by hard timeout [ 1827.108422] Lustre: 101251:0:(ldlm_lib.c:2403:target_recovery_overseer()) recovery is aborted, evict exports in recovery [ 1827.112235] Lustre: 101251:0:(genops.c:1622:class_disconnect_stale_exports()) lustre-MDT0000: disconnect stale client a93a9eae-614c-4e74-8e02-342cefae2918@192.168.202.38@tcp [ 1827.116866] Lustre: lustre-MDT0000: disconnecting 1 stale clients [ 1827.120668] Lustre: 101251:0:(ldlm_lib.c:1913:abort_req_replay_queue()) @@@ aborted: req@00000000eb6eef9c x1875231577805568/t0(8589934782) o35->a93a9eae-614c-4e74-8e02-342cefae2918@192.168.202.38@tcp:604/0 lens 392/0 e 0 to 0 dl 1788360799 ref 1 fl Complete:/604/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:0 [ 1827.135038] Lustre: lustre-MDT0000-osd: cancel update llog [0x200000400:0x1:0x0] [ 1827.142806] Lustre: lustre-MDT0001-osp-MDT0000: cancel update llog [0x240000401:0x1:0x0] [ 1827.150585] Lustre: lustre-MDT0000: Denying connection for new client a93a9eae-614c-4e74-8e02-342cefae2918 (at 192.168.202.38@tcp), waiting for 3 known clients (2 recovered, 0 in progress, and 1 evicted) to recover in 0:00 [ 1827.161057] Lustre: lustre-MDT0000-osp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 1827.163659] Lustre: Skipped 2 previous similar messages [ 1827.165169] Lustre: lustre-MDT0000: Recovery over after 1:00, of 3 clients 2 recovered and 1 was evicted. [ 1827.190792] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:49 to 0x2c0000401:545) [ 1827.195952] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:80 to 0x280000401:577) [ 1834.980614] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1834.981808] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1834.986511] Lustre: Skipped 4 previous similar messages [ 1834.989752] Lustre: Skipped 7 previous similar messages [ 1849.311359] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 1849.369203] Lustre: server umount lustre-OST0000 complete [ 1850.338105] LustreError: 100293:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1850.359402] LustreError: 100293:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 37 previous similar messages [ 1853.411481] LustreError: lustre-OST0001-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1858.523320] Lustre: server umount lustre-OST0001 complete [ 1867.335179] Lustre: server umount lustre-MDT0000 complete [ 1869.941665] LustreError: 95010:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788360831 with bad export cookie 4128200608092844136 [ 1869.943113] LustreError: MGC192.168.202.138@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1869.947022] LustreError: 95010:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 1870.175899] Lustre: server umount lustre-MDT0001 complete [ 1876.016218] Lustre: DEBUG MARKER: == conf-sanity test 85: osd_ost init: fail ea_fid_set ==== 10:53:55 (1788360835) [ 1881.385197] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1881.388536] Lustre: *** cfs_fail_loc=197, val=0*** [ 1896.863957] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 1901.986431] LustreError: 103228:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.202.138@tcp: failed processing log, type 4: rc = -110 [ 1907.116675] Lustre: *** cfs_fail_loc=197, val=0*** [ 1907.119203] Lustre: Skipped 65 previous similar messages [ 1931.657418] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 1936.473353] Lustre: DEBUG MARKER: oleg238-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1937.969218] Lustre: server umount lustre-OST0000 complete [ 1943.570078] Lustre: DEBUG MARKER: == conf-sanity test 86: Replacing mkfs.lustre -G option == 10:55:03 (1788360903) [ 1961.128608] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1961.386148] LustreError: 104847:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1961.401116] LustreError: 104847:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 1964.571580] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 1971.351387] Lustre: DEBUG MARKER: oleg238-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1974.114860] LustreError: 104878:0:(osp_object.c:618:osp_attr_get()) lustre-MDT0001-osp-MDT0000: osp_attr_get update error [0x200000009:0x1:0x0]: rc = -5 [ 1974.120592] LustreError: 104878:0:(lod_sub_object.c:917:lod_sub_prep_llog()) lustre-MDT0000-mdtlov: can't get id from catalogs: rc = -5 [ 1974.126149] LustreError: 104878:0:(lod_dev.c:511:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 12, retries 0, failed: rc = -5 [ 1974.332681] Lustre: server umount lustre-MDT0000 complete [ 1994.517599] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_hostid [ 2001.662530] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing load_modules_local [ 2027.332686] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2027.479161] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2027.498904] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2027.553425] Lustre: lustre-MDT0000: new disk, initializing [ 2027.608292] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2027.616352] Lustre: Skipped 2 previous similar messages [ 2027.629199] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2030.755154] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 2038.368090] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2038.443284] Lustre: 109048:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2038.587288] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2041.438917] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 2047.826839] Lustre: DEBUG MARKER: oleg238-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2051.921957] Lustre: DEBUG MARKER: oleg238-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2056.390834] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2056.557803] Lustre: 111019:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2060.816126] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 2063.450328] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2067.129963] Lustre: DEBUG MARKER: oleg238-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2070.910535] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2071.054989] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2074.282840] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2074.426900] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2078.694707] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2078.700045] LustreError: Skipped 2 previous similar messages [ 2078.703140] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2078.711492] Lustre: Skipped 4 previous similar messages [ 2078.715611] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2078.718734] Lustre: Skipped 11 previous similar messages [ 2081.783098] Lustre: server umount lustre-OST0000 complete [ 2090.461773] Lustre: server umount lustre-MDT0000 complete [ 2093.183982] LustreError: 109041:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788361054 with bad export cookie 4128200608092854853 [ 2093.184647] LustreError: MGC192.168.202.138@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2093.190786] LustreError: 109041:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 2093.368644] Lustre: server umount lustre-MDT0001 complete [ 2100.057778] Lustre: DEBUG MARKER: == conf-sanity test 87: check if MDT inode can hold EAs with N stripes properly ========================================================== 10:57:39 (1788361059) [ 2106.297927] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing unload_modules_local [ 2108.052579] Key type lgssc unregistered [ 2108.251785] LNet: 113505:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2108.257836] LNetError: 113505:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2109.293726] LNet: Removed LNI 192.168.202.138@tcp [ 2109.984190] Key type .llcrypt unregistered [ 2109.986762] Key type ._llcrypt unregistered [ 2121.836415] Key type ._llcrypt registered [ 2121.839082] Key type .llcrypt registered [ 2121.893372] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_hostid [ 2132.487826] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing load_modules_local [ 2133.083690] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2133.107378] alg: No test for adler32 (adler32-zlib) [ 2134.047853] Lustre: Lustre: Build Version: 2.17.56_1_g896c54c [ 2134.205126] LNet: Added LNI 192.168.202.138@tcp [8/256/0/180] [ 2135.823218] Key type lgssc registered [ 2136.587374] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2162.760586] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 2162.783281] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2163.952919] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2164.007268] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2164.078956] Lustre: lustre-MDT0000: new disk, initializing [ 2164.132591] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2164.145090] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2167.322418] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 2175.007216] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2175.277393] Lustre: lustre-OST0000: new disk, initializing [ 2175.282808] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 2175.287422] Lustre: Skipped 1 previous similar message [ 2175.290091] Lustre: 117652:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2175.350165] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2177.817229] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 2177.825072] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 2177.842192] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 2178.657565] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 2185.110623] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2185.194801] Lustre: lustre-OST0001: new disk, initializing [ 2185.197328] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 2185.204624] Lustre: 118516:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2185.241033] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 2187.942389] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:1:ost [ 2187.960846] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:1:ost] [ 2187.991685] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x280000400 [ 2189.450313] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 2198.499685] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2198.502894] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2198.515582] Lustre: Skipped 1 previous similar message [ 2198.527417] Lustre: Skipped 1 previous similar message [ 2201.371260] Lustre: server umount lustre-MDT0000 complete [ 2204.703888] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: (null) [ 2207.160857] LustreError: 118515:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788361168 with bad export cookie 6363981362930054779 [ 2207.166433] LustreError: 118515:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 2207.166680] LustreError: MGC192.168.202.138@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2207.206715] Lustre: server umount lustre-OST0000 complete [ 2209.367434] Lustre: server umount lustre-OST0001 complete [ 2214.113715] Lustre: DEBUG MARKER: == conf-sanity test 88a: tunefs.lustre persists a --mountfsoptions-only change ========================================================== 10:59:34 (1788361174) [ 2227.406758] Lustre: DEBUG MARKER: == conf-sanity test 88b: check the default mount options can be overridden ========================================================== 10:59:47 (1788361187) [ 2232.924938] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing load_modules_local [ 2246.229216] Lustre: DEBUG MARKER: == conf-sanity test 89: check tunefs --param and --erase-param{s} options ========================================================== 11:00:06 (1788361206) [ 2268.471609] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_hostid [ 2274.111202] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing load_modules_local [ 2298.091288] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2298.258860] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2298.274598] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2298.322577] Lustre: lustre-MDT0000: new disk, initializing [ 2298.363811] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2298.379260] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2301.604107] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 2308.840652] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2308.947859] Lustre: 126254:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2309.008806] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 2309.011731] Lustre: Skipped 1 previous similar message [ 2309.061823] Lustre: lustre-MDT0001: new disk, initializing [ 2309.132941] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2309.158768] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 2309.163606] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2312.010500] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 2318.120552] Lustre: DEBUG MARKER: oleg238-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2321.585587] Lustre: DEBUG MARKER: oleg238-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2325.834415] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2325.991414] Lustre: lustre-OST0000: new disk, initializing [ 2325.994305] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 2325.998610] Lustre: 128236:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2326.062538] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2329.930277] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 2332.234293] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 2332.244485] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2332.307374] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2336.039103] Lustre: DEBUG MARKER: oleg238-client.virtnet: executing wait_import_state_mount (FULL|IDLE) failover.node=192.0.2.254@tcp0,osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2338.922156] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2339.055585] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2341.656101] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2341.778587] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2348.003930] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2348.004111] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2348.008911] LustreError: Skipped 1 previous similar message [ 2348.016802] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2348.024091] Lustre: Skipped 1 previous similar message [ 2349.178520] Lustre: server umount lustre-OST0000 complete [ 2353.122952] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2353.132979] Lustre: Skipped 2 previous similar messages [ 2353.135174] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2353.138346] Lustre: Skipped 1 previous similar message [ 2357.583565] Lustre: server umount lustre-MDT0000 complete [ 2359.994612] LustreError: 126248:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788361321 with bad export cookie 6363981362930056004 [ 2360.000225] LustreError: MGC192.168.202.138@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2360.003025] LustreError: 126248:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 2360.224636] Lustre: server umount lustre-MDT0001 complete [ 2365.831297] Lustre: DEBUG MARKER: == conf-sanity test 90a: check max_mod_rpcs_in_flight is enforced ========================================================== 11:02:05 (1788361325) [ 2369.937545] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2370.234525] LustreError: 130646:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2370.248191] LustreError: 130646:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [ 2370.304203] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2372.987358] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 2375.652379] LustreError: 130647:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2378.615830] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2381.756714] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 2386.828625] Lustre: DEBUG MARKER: oleg238-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2390.641783] Lustre: DEBUG MARKER: oleg238-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2394.787427] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2398.547835] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 2403.339191] Lustre: DEBUG MARKER: oleg238-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2410.265423] Lustre: *** cfs_fail_loc=159, val=0*** [ 2410.269601] Lustre: Skipped 1 previous similar message [ 2426.848481] Lustre: lustre-MDT0000: Client 283e20ef-5a47-4191-962d-2dfd3cd268d7 (at 192.168.202.38@tcp) reconnecting [ 2427.786425] Lustre: *** cfs_fail_loc=159, val=0*** [ 2427.792231] Lustre: Skipped 4 previous similar messages [ 2443.223523] Lustre: lustre-MDT0000: Client 283e20ef-5a47-4191-962d-2dfd3cd268d7 (at 192.168.202.38@tcp) reconnecting [ 2460.642963] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2460.646414] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2460.652828] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2460.657479] Lustre: Skipped 1 previous similar message [ 2460.666754] Lustre: Skipped 1 previous similar message [ 2465.764613] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2465.772337] Lustre: Skipped 1 previous similar message [ 2465.869720] Lustre: server umount lustre-OST0000 complete [ 2470.880428] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2470.883157] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2470.901488] Lustre: Skipped 1 previous similar message [ 2474.382712] Lustre: server umount lustre-MDT0000 complete [ 2476.000557] LustreError: 130646:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2476.016313] LustreError: 130646:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 2476.608253] LustreError: 132487:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788361438 with bad export cookie 6363981362930056886 [ 2476.610404] LustreError: MGC192.168.202.138@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2476.615591] LustreError: 132487:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 2476.802300] Lustre: server umount lustre-MDT0001 complete [ 2483.502942] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing unload_modules_local [ 2485.285087] Key type lgssc unregistered [ 2485.464469] LNet: 134418:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2485.469644] LNetError: 134418:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2485.492893] LNet: Removed LNI 192.168.202.138@tcp [ 2486.053734] Key type .llcrypt unregistered [ 2486.055735] Key type ._llcrypt unregistered [ 2500.720824] Key type ._llcrypt registered [ 2500.722219] Key type .llcrypt registered [ 2500.786592] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing load_modules_local [ 2501.438293] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2501.456088] alg: No test for adler32 (adler32-zlib) [ 2502.364675] Lustre: Lustre: Build Version: 2.17.56_1_g896c54c [ 2502.495643] LNet: Added LNI 192.168.202.138@tcp [8/256/0/180] [ 2504.103566] Key type lgssc registered [ 2504.717892] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2508.734825] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 2508.745643] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2509.991234] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2511.889724] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 2517.034925] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2517.272386] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2519.216184] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 2522.502502] Lustre: DEBUG MARKER: oleg238-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2525.128826] Lustre: DEBUG MARKER: oleg238-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2528.318421] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2528.484243] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2531.830672] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 2534.634172] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:42 to 0x280000401:65) [ 2536.085170] Lustre: DEBUG MARKER: oleg238-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2547.374331] Lustre: *** cfs_fail_loc=159, val=0*** [ 2563.516777] Lustre: lustre-MDT0000: Client 93dc165c-859f-4564-bb6f-30d5e7593a15 (at 192.168.202.38@tcp) reconnecting [ 2564.908338] Lustre: *** cfs_fail_loc=159, val=0*** [ 2580.428768] Lustre: lustre-MDT0001: Client 93dc165c-859f-4564-bb6f-30d5e7593a15 (at 192.168.202.38@tcp) reconnecting [ 2581.409222] Lustre: *** cfs_fail_loc=159, val=0*** [ 2581.413598] Lustre: Skipped 4 previous similar messages [ 2597.843658] Lustre: lustre-MDT0001: Client 93dc165c-859f-4564-bb6f-30d5e7593a15 (at 192.168.202.38@tcp) reconnecting [ 2612.595562] Lustre: 138867:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 2614.318254] Lustre: *** cfs_fail_loc=159, val=0*** [ 2614.321804] Lustre: Skipped 3 previous similar messages [ 2630.604611] Lustre: lustre-MDT0000: Client 6eabc08c-9844-43cd-af4b-e55cfd409965 (at 192.168.202.38@tcp) reconnecting [ 2631.561658] Lustre: *** cfs_fail_loc=159, val=0*** [ 2631.563977] Lustre: Skipped 14 previous similar messages [ 2648.008879] Lustre: lustre-MDT0000: Client 6eabc08c-9844-43cd-af4b-e55cfd409965 (at 192.168.202.38@tcp) reconnecting [ 2648.883502] Lustre: 139159:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 2668.000830] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2668.009315] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2668.021047] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2670.049845] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2670.053144] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2670.058129] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2672.380301] Lustre: server umount lustre-OST0000 complete [ 2675.171175] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2675.172824] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2675.179754] Lustre: Skipped 1 previous similar message [ 2675.184484] Lustre: Skipped 1 previous similar message [ 2680.290653] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2680.294394] Lustre: Skipped 2 previous similar messages [ 2680.707326] Lustre: server umount lustre-MDT0000 complete [ 2682.745148] LustreError: 135793:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788361644 with bad export cookie 15375947971301512688 [ 2682.746262] LustreError: MGC192.168.202.138@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2682.760399] LustreError: 135793:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2682.907729] Lustre: server umount lustre-MDT0001 complete [ 2688.742021] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing unload_modules_local [ 2690.119519] Key type lgssc unregistered [ 2690.273539] LNet: 140236:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2690.278620] LNetError: 140236:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2690.293583] LNet: Removed LNI 192.168.202.138@tcp [ 2690.799784] Key type .llcrypt unregistered [ 2690.801132] Key type ._llcrypt unregistered [ 2703.922689] Key type ._llcrypt registered [ 2703.924130] Key type .llcrypt registered [ 2703.971657] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing load_modules_local [ 2704.392869] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2704.432761] alg: No test for adler32 (adler32-zlib) [ 2705.358296] Lustre: Lustre: Build Version: 2.17.56_1_g896c54c [ 2705.516688] LNet: Added LNI 192.168.202.138@tcp [8/256/0/180] [ 2707.127177] Key type lgssc registered [ 2707.864472] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2711.962507] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 2711.972558] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2713.208480] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2715.166299] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 2719.142409] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2719.343329] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2721.233211] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 2724.843601] Lustre: DEBUG MARKER: oleg238-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2727.590495] Lustre: DEBUG MARKER: oleg238-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2730.854555] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2731.048342] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2734.230584] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 2738.216474] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:129) [ 2738.375626] Lustre: DEBUG MARKER: oleg238-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2744.364992] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:97) [ 2745.876578] Lustre: 144208:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 2747.106867] Lustre: 144256:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 2747.791377] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 in log params [ 2749.510513] Lustre: 144403:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre/mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 (mode = 0) failed: rc = -17 [ 2750.018971] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=7 in log params [ 2750.022266] Lustre: Skipped 1 previous similar message [ 2750.509658] Lustre: 144501:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 2753.506382] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2753.509640] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2753.515967] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2754.531500] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2754.532544] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2754.533996] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2758.393949] Lustre: server umount lustre-OST0000 complete [ 2764.772234] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2764.779426] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2764.784306] Lustre: Skipped 1 previous similar message [ 2764.791030] Lustre: Skipped 2 previous similar messages [ 2766.590578] Lustre: server umount lustre-MDT0000 complete [ 2768.572910] LustreError: 141609:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788361730 with bad export cookie 14558653645942770664 [ 2768.581051] LustreError: MGC192.168.202.138@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2768.755545] Lustre: server umount lustre-MDT0001 complete [ 2774.410472] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing unload_modules_local [ 2775.937923] Key type lgssc unregistered [ 2776.109623] LNet: 145557:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2776.112692] LNetError: 145557:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2776.124672] LNet: Removed LNI 192.168.202.138@tcp [ 2776.560900] Key type .llcrypt unregistered [ 2776.562096] Key type ._llcrypt unregistered [ 2790.489325] Key type ._llcrypt registered [ 2790.493646] Key type .llcrypt registered [ 2790.546614] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing load_modules_local [ 2790.999884] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2791.012189] alg: No test for adler32 (adler32-zlib) [ 2791.915697] Lustre: Lustre: Build Version: 2.17.56_1_g896c54c [ 2792.025396] LNet: Added LNI 192.168.202.138@tcp [8/256/0/180] [ 2793.647139] Key type lgssc registered [ 2794.319550] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2799.084775] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 2799.103822] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2800.341641] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2802.537736] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 2807.236711] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2807.418788] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2809.508376] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 2814.684729] Lustre: DEBUG MARKER: oleg238-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2818.515254] Lustre: DEBUG MARKER: oleg238-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2822.908617] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2823.109903] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2825.195814] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:161) [ 2827.056964] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 2831.838996] Lustre: DEBUG MARKER: oleg238-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2832.363139] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:129) [ 2836.035350] Lustre: *** cfs_fail_loc=159, val=0*** [ 2851.260033] Lustre: lustre-MDT0000: Client a9e790ae-3589-4fdb-90f7-069f52dbab8b (at 192.168.202.38@tcp) reconnecting [ 2855.905013] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2855.910081] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2855.919900] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2857.953646] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2857.953852] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2857.960100] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2859.398799] Lustre: server umount lustre-OST0000 complete [ 2863.073722] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2863.080496] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2863.086082] Lustre: Skipped 2 previous similar messages [ 2867.743467] Lustre: server umount lustre-MDT0000 complete [ 2868.192340] LustreError: 146950:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2868.200315] LustreError: 146950:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [ 2869.629810] LustreError: 146930:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788361831 with bad export cookie 14443377293248658242 [ 2869.633336] LustreError: MGC192.168.202.138@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2869.636282] LustreError: 146930:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2869.823533] Lustre: server umount lustre-MDT0001 complete [ 2874.976783] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing unload_modules_local [ 2876.407832] Key type lgssc unregistered [ 2876.572468] LNet: 150607:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2876.575836] LNetError: 150607:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2876.586383] LNet: Removed LNI 192.168.202.138@tcp [ 2877.002595] Key type .llcrypt unregistered [ 2877.004080] Key type ._llcrypt unregistered [ 2889.248051] Key type ._llcrypt registered [ 2889.249964] Key type .llcrypt registered [ 2889.300653] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing load_modules_local [ 2889.685712] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2889.696515] alg: No test for adler32 (adler32-zlib) [ 2890.576823] Lustre: Lustre: Build Version: 2.17.56_1_g896c54c [ 2890.682663] LNet: Added LNI 192.168.202.138@tcp [8/256/0/180] [ 2892.279153] Key type lgssc registered [ 2892.769375] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2896.283597] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 2896.295457] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2897.538203] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2899.568357] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 2903.745080] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2903.947408] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2905.905830] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 2909.211889] Lustre: DEBUG MARKER: oleg238-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2911.532273] Lustre: DEBUG MARKER: oleg238-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2914.431275] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2914.612444] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2917.412069] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 2920.897605] Lustre: DEBUG MARKER: oleg238-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2922.791472] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:170 to 0x280000401:193) [ 2928.932280] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:161) [ 2945.691054] Lustre: lustre-OST0000: evicting 4352da8c-d62f-4f5a-bb73-0b1065ca3446 (at 192.168.202.38@tcp) by administrative request [ 2945.695689] Lustre: Skipped 1 previous similar message [ 2974.175764] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2974.179353] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2974.184335] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2975.203633] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2975.205427] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2975.212289] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2975.214173] Lustre: Skipped 1 previous similar message [ 2976.287863] Lustre: server umount lustre-OST0000 complete [ 2980.320992] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2980.325489] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2980.333018] Lustre: Skipped 1 previous similar message [ 2980.342841] Lustre: Skipped 1 previous similar message [ 2983.979478] Lustre: server umount lustre-MDT0000 complete [ 2985.440208] LustreError: 152017:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2985.449338] LustreError: 152017:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [ 2985.559653] LustreError: 151998:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788361947 with bad export cookie 4612564915449369224 [ 2985.564162] LustreError: MGC192.168.202.138@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2985.568213] LustreError: 151998:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 2985.699419] Lustre: server umount lustre-MDT0001 complete [ 2990.085108] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing unload_modules_local [ 2991.471909] Key type lgssc unregistered [ 2991.627209] LNet: 156106:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2991.630548] LNetError: 156106:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2991.641365] LNet: Removed LNI 192.168.202.138@tcp [ 2992.013669] Key type .llcrypt unregistered [ 2992.016506] Key type ._llcrypt unregistered [ 3020.624811] Key type ._llcrypt registered [ 3020.626314] Key type .llcrypt registered [ 3020.661816] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing load_modules_local [ 3020.991366] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3021.026030] alg: No test for adler32 (adler32-zlib) [ 3021.884331] Lustre: Lustre: Build Version: 2.17.56_1_g896c54c [ 3021.977427] LNet: Added LNI 192.168.202.138@tcp [8/256/0/180] [ 3023.559177] Key type lgssc registered [ 3023.990140] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3026.901772] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3026.909818] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3028.078493] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3029.733568] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 3033.235160] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3033.386751] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3035.010779] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 3038.046626] Lustre: DEBUG MARKER: oleg238-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3040.506872] Lustre: DEBUG MARKER: oleg238-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3043.688206] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3043.880420] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3046.404326] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 3049.464130] Lustre: DEBUG MARKER: oleg238-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3053.097376] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:195 to 0x280000401:225) [ 3058.211760] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:193) [ 3067.876000] LustreError: lustre-OST0000-osc-MDT0000: operation ost_destroy to node 0@lo failed: rc = -107 [ 3067.878732] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3067.883980] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3068.385522] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3068.386454] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3068.390491] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3073.505920] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3073.508057] Lustre: Skipped 2 previous similar messages [ 3078.111110] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 3078.167992] Lustre: server umount lustre-OST0000 complete [ 3079.648126] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3079.654191] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3079.660084] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3083.744141] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3083.744638] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3085.785792] Lustre: server umount lustre-MDT0000 complete [ 3087.232444] LustreError: 158546:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788362048 with bad export cookie 4853216888737575304 [ 3087.237255] LustreError: 158546:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 3087.237987] LustreError: MGC192.168.202.138@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3087.363241] Lustre: server umount lustre-MDT0001 complete [ 3091.396704] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing unload_modules_local [ 3092.519434] Key type lgssc unregistered [ 3092.657342] LNet: 162128:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3092.661497] LNetError: 162128:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3092.672327] LNet: Removed LNI 192.168.202.138@tcp [ 3093.008870] Key type .llcrypt unregistered [ 3093.010027] Key type ._llcrypt unregistered [ 3102.923967] Key type ._llcrypt registered [ 3102.926204] Key type .llcrypt registered [ 3102.971268] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing load_modules_local [ 3103.310957] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3103.349053] alg: No test for adler32 (adler32-zlib) [ 3104.214169] Lustre: Lustre: Build Version: 2.17.56_1_g896c54c [ 3104.309024] LNet: Added LNI 192.168.202.138@tcp [8/256/0/180] [ 3105.895508] Key type lgssc registered [ 3106.436652] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3112.278935] Lustre: DEBUG MARKER: == conf-sanity test 100: check lshowmount lists MGS, MDT, OST and 0@lo ========================================================== 11:14:32 (1788362072) [ 3118.193177] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_hostid [ 3121.549899] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing load_modules_local [ 3133.593837] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3133.599592] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3134.687120] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3134.701113] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3134.739912] Lustre: lustre-MDT0000: new disk, initializing [ 3134.779056] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3134.789519] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3136.190246] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 3140.490923] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3140.531057] Lustre: 166359:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 3140.555647] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 3140.558390] Lustre: Skipped 1 previous similar message [ 3140.596927] Lustre: lustre-MDT0001: new disk, initializing [ 3140.629416] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3140.644878] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 3140.652313] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3142.008661] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 3145.830285] Lustre: DEBUG MARKER: oleg238-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3147.777513] Lustre: DEBUG MARKER: oleg238-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3150.077228] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3150.181300] Lustre: lustre-OST0000: new disk, initializing [ 3150.183331] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3150.187120] Lustre: 168329:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3150.216023] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3152.138408] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 3152.443982] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 3152.448086] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 3152.487911] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3155.665632] Lustre: DEBUG MARKER: oleg238-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3162.593473] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3162.594247] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3162.597198] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3162.600460] Lustre: Skipped 1 previous similar message [ 3162.604034] Lustre: Skipped 1 previous similar message [ 3164.886817] Lustre: server umount lustre-OST0000 complete [ 3166.175850] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3166.179144] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3166.184879] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3167.712132] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3167.712633] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3172.259109] Lustre: server umount lustre-MDT0000 complete [ 3173.581075] LustreError: 166353:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788362135 with bad export cookie 8453953655406156473 [ 3173.583901] LustreError: MGC192.168.202.138@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3173.585472] LustreError: 166353:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 3173.722889] Lustre: server umount lustre-MDT0001 complete [ 3177.401793] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing unload_modules_local [ 3178.454570] Key type lgssc unregistered [ 3178.577394] LNet: 170230:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3178.579917] LNetError: 170230:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3178.586341] LNet: Removed LNI 192.168.202.138@tcp [ 3178.926794] Key type .llcrypt unregistered [ 3178.928220] Key type ._llcrypt unregistered [ 3187.969170] Key type ._llcrypt registered [ 3187.970415] Key type .llcrypt registered [ 3188.006365] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing load_modules_local [ 3188.393865] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3188.404509] alg: No test for adler32 (adler32-zlib) [ 3189.258382] Lustre: Lustre: Build Version: 2.17.56_1_g896c54c [ 3189.336823] LNet: Added LNI 192.168.202.138@tcp [8/256/0/180] [ 3190.911164] Key type lgssc registered [ 3191.239791] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3193.805446] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3193.810654] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3194.943965] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3196.241573] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 3198.927114] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3199.044923] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3200.374077] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 3202.678833] Lustre: DEBUG MARKER: oleg238-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3204.482391] Lustre: DEBUG MARKER: oleg238-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3206.807859] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3206.931644] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3208.968576] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 3211.400756] Lustre: DEBUG MARKER: oleg238-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3220.262236] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:3 to 0x280000401:33) [ 3224.377788] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 3224.385740] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3224.392677] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 3224.395864] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3224.402146] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 3224.735811] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3225.080414] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 3225.083766] Lustre: Skipped 1 previous similar message [ 3225.089920] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3225.095383] Lustre: Skipped 1 previous similar message [ 3225.097021] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 3225.099144] Lustre: Skipped 1 previous similar message [ 3225.101034] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3225.104556] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 3225.106772] Lustre: Skipped 1 previous similar message [ 3225.408466] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3225.734867] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3226.043452] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3226.354811] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 3226.357862] Lustre: Skipped 3 previous similar messages [ 3226.364055] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3226.368228] Lustre: Skipped 3 previous similar messages [ 3226.370476] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 3226.373491] Lustre: Skipped 3 previous similar messages [ 3226.375573] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3226.379483] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 3226.381768] Lustre: Skipped 3 previous similar messages [ 3226.680761] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3227.000540] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3227.309880] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3227.689646] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3228.011776] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3228.330147] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3228.648887] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 3228.653644] Lustre: Skipped 6 previous similar messages [ 3228.660322] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3228.667557] Lustre: Skipped 6 previous similar messages [ 3228.669527] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 3228.671787] Lustre: Skipped 6 previous similar messages [ 3228.673407] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3228.677335] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 3228.679959] Lustre: Skipped 6 previous similar messages [ 3229.016879] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3229.354637] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3229.655098] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3230.003080] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3230.357310] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3230.676817] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3231.012090] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3231.356195] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3231.692061] LustreError: 171642:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 3231.695122] LustreError: 171642:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 3231.697398] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3232.014117] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3232.341784] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3232.685275] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 3232.687307] Lustre: Skipped 11 previous similar messages [ 3232.692381] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3232.696295] Lustre: Skipped 11 previous similar messages [ 3232.697981] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 3232.700576] Lustre: Skipped 11 previous similar messages [ 3232.702218] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3232.705607] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 3232.708094] Lustre: Skipped 11 previous similar messages [ 3233.041250] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3233.347290] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3233.649292] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3233.972752] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3234.300865] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3234.601142] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3234.913917] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3235.229517] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3235.570822] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3235.884219] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3236.189706] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3236.486530] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3236.798626] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3237.103295] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3237.443575] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3237.755834] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3238.087671] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3238.383318] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3238.671652] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3238.960567] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3239.267890] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3239.572319] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3239.867467] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3240.164124] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3240.508275] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3240.821912] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 3240.823823] Lustre: Skipped 25 previous similar messages [ 3240.828624] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3240.831996] Lustre: Skipped 25 previous similar messages [ 3240.833795] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 3240.836283] Lustre: Skipped 25 previous similar messages [ 3240.837507] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3240.840894] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 3240.842900] Lustre: Skipped 25 previous similar messages [ 3241.149333] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3241.458732] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3241.757249] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3242.044184] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3242.339605] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3242.618101] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3242.909686] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3243.197271] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3243.486248] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3243.778222] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3244.069563] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3244.364325] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3244.652035] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3244.952214] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3245.260516] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3245.566753] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3245.884230] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3246.204316] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3246.532105] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3246.837668] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3247.149414] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3247.450704] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3247.765354] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3248.077132] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3248.382391] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3248.678983] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3249.003304] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3249.312867] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3249.617964] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3249.917503] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3250.232918] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3250.516371] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3250.811383] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3251.101704] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3251.402884] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3251.693805] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3252.001863] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3252.305333] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3252.609685] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3252.895146] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3253.187045] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3253.499580] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3253.822772] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3254.135033] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3254.449552] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3254.759423] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3255.079509] LustreError: 171642:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 3255.082599] LustreError: 171642:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 3255.082859] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3255.377913] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3255.682067] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3255.992638] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3256.310753] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3256.609948] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3256.909636] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 3256.911713] Lustre: Skipped 52 previous similar messages [ 3256.916795] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3256.920044] Lustre: Skipped 52 previous similar messages [ 3256.921479] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 3256.924381] Lustre: Skipped 52 previous similar messages [ 3256.926271] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3256.930572] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 3256.932707] Lustre: Skipped 52 previous similar messages [ 3257.216752] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3257.525165] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3257.825164] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3258.115981] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3258.417211] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3258.751558] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3259.068340] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3259.392516] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3259.694669] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3260.014629] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3260.333231] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3260.647756] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3260.942374] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3261.252076] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3261.559160] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3261.869889] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3262.180688] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3262.520262] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3262.874122] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3263.188235] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3263.527075] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3263.851476] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3264.170264] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3264.482767] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3264.798885] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3265.119247] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3265.427918] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3265.764691] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3266.077210] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3266.417852] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3266.726322] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3267.070352] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3267.389987] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3267.724256] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3268.045337] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3268.386864] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3268.728766] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3269.121377] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3269.475442] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3269.796880] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3270.143848] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3270.473733] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3270.812143] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3271.137292] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3271.479717] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3271.846076] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3272.205466] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3272.575632] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3272.938302] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3273.294523] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3273.623272] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3273.927055] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3274.220044] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3274.502361] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3274.801796] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3275.123194] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3275.435209] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3275.754710] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3276.068107] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3276.412447] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3276.746214] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3277.073083] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3277.389868] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3277.697240] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3277.988762] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3278.288312] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3278.556158] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3278.843619] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3279.123896] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3279.399514] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3279.676386] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3279.972909] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3280.288398] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3280.605420] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3280.924814] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3281.269972] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3281.580287] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3281.874508] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3282.174716] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3282.496731] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3282.781875] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3283.074591] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3283.381047] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3283.696461] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3283.997249] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3284.298266] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3284.618018] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3284.959585] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3285.263707] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3285.576365] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3285.887193] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3286.182770] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3286.464988] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3286.767313] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3287.046937] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3287.359339] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3287.655676] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3287.946964] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3288.254377] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3288.559743] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3288.862193] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3289.169525] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 3289.171568] Lustre: Skipped 101 previous similar messages [ 3289.176089] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3289.179262] Lustre: Skipped 101 previous similar messages [ 3289.180676] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 3289.182295] Lustre: Skipped 101 previous similar messages [ 3289.183480] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3289.186639] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 3289.188402] Lustre: Skipped 101 previous similar messages [ 3289.466959] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3289.786029] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3290.105055] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3290.435318] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3290.746751] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3291.081220] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3291.393343] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3291.703266] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3292.018500] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3292.357502] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3292.689602] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3293.010262] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3293.347215] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3293.692365] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3294.034217] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3294.382957] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3294.699710] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3294.996336] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3295.314338] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3295.619688] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3295.916060] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3296.216878] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3296.525927] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3296.864748] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3297.194770] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3297.500544] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3297.805127] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3298.124630] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3298.433647] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3298.730769] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3299.024767] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3299.360112] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3299.663353] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3299.983567] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3300.284059] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3300.602798] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3300.905032] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3301.213500] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3301.534031] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3301.870896] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3302.185054] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3302.492159] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3302.789763] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3303.111459] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3303.410169] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3303.723063] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3304.022353] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3304.345883] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3304.659664] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3304.973776] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3305.294947] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3305.624025] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3305.943201] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3306.263557] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3306.583450] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3306.924937] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3307.236380] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3307.566808] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3307.892384] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3308.252589] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3308.562091] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3308.881887] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3309.185527] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3309.520793] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3309.863058] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3310.184653] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3310.513029] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3310.829992] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3311.157784] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3311.462636] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3311.777791] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3312.139174] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3312.474668] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3312.817387] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3313.143798] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3313.484384] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3313.815177] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3314.123724] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3314.426888] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3314.740785] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3315.076905] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3315.398874] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3315.715114] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3316.083931] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3316.428088] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3316.767314] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3317.135903] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3317.504202] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3317.843553] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3318.180782] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3318.541106] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3318.881976] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3319.217758] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3319.566508] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3319.892733] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3320.213675] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3320.547597] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3320.857094] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3321.195188] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3321.533663] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3321.877715] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3322.225237] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3322.556904] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3322.903283] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3323.255038] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3323.591256] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3323.909291] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3324.232485] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3324.568673] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3324.895606] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3325.221462] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3325.540965] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3325.884631] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3326.215865] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3326.539282] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3326.880162] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3327.257979] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3327.606697] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3327.933085] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3328.245794] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3328.603521] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3328.980473] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3329.329165] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3329.689135] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3330.028280] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3330.354617] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3330.679993] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3331.033563] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3331.359739] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3331.735475] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3332.134562] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3332.538521] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3332.893194] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3333.270035] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3333.624923] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3333.946167] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3334.312303] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3334.627320] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3334.943723] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3335.261838] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3335.618037] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3335.985952] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3336.343273] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3336.712529] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3337.096312] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3337.441102] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3337.759425] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3338.126407] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3338.486599] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3338.833457] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3339.192284] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3339.541982] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3339.904239] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3340.305140] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3340.670528] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3341.007628] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3341.347464] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3341.692048] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3342.046285] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3342.407696] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3342.750617] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3343.140480] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3343.498695] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3343.871955] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3344.214881] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3344.570767] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3344.924498] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3345.295242] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3345.617195] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3345.937569] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3346.276157] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3346.628262] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3346.976949] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3347.356571] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3347.684070] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3348.025576] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3348.381253] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3348.725070] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3349.051168] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3349.406926] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3349.772823] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3350.161501] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3350.542539] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3350.903439] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3351.279574] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3351.630296] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3351.991946] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3352.444555] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3352.844361] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3353.204585] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 3353.207826] Lustre: Skipped 189 previous similar messages [ 3353.214913] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3353.220531] Lustre: Skipped 189 previous similar messages [ 3353.222860] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 3353.226155] Lustre: Skipped 189 previous similar messages [ 3353.228280] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3353.233099] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 3353.235678] Lustre: Skipped 189 previous similar messages [ 3353.619570] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3353.989667] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3354.354108] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3354.720036] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3355.082397] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3355.443689] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3355.790524] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3356.133522] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3356.491644] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3356.851386] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3357.218619] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3357.601281] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3357.956924] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3358.334964] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3358.704192] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3359.068307] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3359.437317] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3359.818768] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3360.164847] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3360.496784] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3360.852395] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3361.197134] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3361.518328] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3361.885593] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3362.268443] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3362.664714] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3363.003958] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3363.330957] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3363.669875] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3364.008207] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3364.357683] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3364.670818] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3364.989695] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3365.325540] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3365.674252] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3366.013564] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3366.341485] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3366.682606] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3367.064858] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3367.440298] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3367.777982] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3368.111493] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3368.450037] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3368.781051] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3369.125485] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3369.458170] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3369.774284] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3370.101966] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3370.441879] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3370.770581] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3371.101279] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3371.435492] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3371.764284] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3372.096436] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3372.465309] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3372.789861] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3373.113203] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3373.431706] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3373.781418] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3374.114177] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3374.447161] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3374.779457] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3375.128374] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3375.472624] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3375.810541] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3376.148866] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3376.464668] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3376.783076] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3377.138564] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3377.538318] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3377.901849] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3378.236788] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3378.567934] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3378.922904] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3379.255473] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3379.602828] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3379.948715] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3380.287217] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3380.619250] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3380.941801] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3381.275992] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3381.592583] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3381.920486] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3382.261331] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3382.629172] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3382.950285] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3383.260174] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3383.577789] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3383.924249] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3384.241168] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3384.563789] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3384.895678] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3385.233844] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3385.561563] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3385.908149] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3386.229212] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3386.576043] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3386.900746] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3387.228644] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3387.600481] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3387.996161] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3388.340558] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3388.671508] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3389.022337] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3389.392586] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3389.706933] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3390.057706] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3390.392654] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3390.741064] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3391.060229] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3391.373801] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3391.699938] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3392.047226] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3392.392773] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3392.798312] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3393.153522] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3393.515259] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3393.837220] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3394.154573] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3394.486331] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3394.814247] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3395.139544] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3395.452857] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3395.798106] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3396.121691] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3396.445808] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3396.761357] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3397.108198] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3397.462568] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3397.837836] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3398.209779] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3398.572563] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3398.920087] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3399.304666] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3399.642749] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3399.985452] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3400.321912] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3400.670642] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3401.017904] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3401.350054] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3401.689317] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3402.053299] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3402.420727] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3402.800994] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3403.136523] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3403.468224] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3403.822407] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3404.170437] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3404.528474] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3404.868517] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3405.208571] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3405.585566] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3405.925890] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3406.324295] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3406.666050] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3407.004045] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3407.365301] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3407.712112] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3408.050752] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3408.367286] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3408.699360] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3409.042111] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3409.383052] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3409.720768] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3410.054290] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3410.407853] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3410.755215] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3411.101359] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3411.448766] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3411.780938] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3412.140744] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3412.508959] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3412.899138] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3413.282723] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3413.639725] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3413.955256] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3414.287692] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3414.610715] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3414.941516] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3415.268326] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3415.611130] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3415.970950] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3416.308860] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3416.652279] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3416.971321] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3417.304547] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3417.672614] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3418.009262] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3418.334608] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3418.654255] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3418.994582] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3419.298348] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3419.592066] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3419.905753] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3420.262424] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3420.578841] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3420.887308] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3421.187358] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3421.503153] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3421.823700] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3422.116159] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3422.423310] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3422.754811] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3423.069942] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3423.374509] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3423.689070] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3424.016751] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3424.341372] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3424.685750] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3425.040988] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3425.395763] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3425.738857] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3426.072526] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3426.428629] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3426.750359] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3427.082317] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3427.403197] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3427.747342] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3428.061786] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3428.376035] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3428.698389] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3429.013917] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3429.310959] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3429.632166] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3429.951660] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3430.259959] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3430.574380] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3430.890924] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3431.229930] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3431.578749] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3431.886875] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3432.187257] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3432.498286] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3432.851758] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3433.233126] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3433.544592] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3433.854181] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3434.203129] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3434.513335] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3434.856424] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3435.184181] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3435.518238] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3435.850000] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3436.204503] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3436.544836] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3436.903322] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3437.248966] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3437.600347] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3437.969648] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3438.311248] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3438.639362] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3438.981436] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3439.323957] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3439.714044] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3440.053230] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3440.412614] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3440.732816] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3441.065214] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3441.373267] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3441.676724] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3441.989314] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3442.319167] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3442.647118] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3442.991265] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3443.350624] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3443.723627] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3444.063276] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3444.401856] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3444.732448] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3445.080101] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3445.429267] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3445.747430] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3446.054052] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3446.426714] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3446.723354] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3447.027879] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3447.334488] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3447.656501] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3447.951710] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3448.292841] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3448.606827] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3448.934134] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3449.278230] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3449.607399] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3449.903876] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3450.215459] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3450.513831] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3450.808504] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3451.118520] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3451.443738] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3451.758936] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3452.079460] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3452.406807] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3452.749279] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3453.050202] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3453.377403] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3453.673404] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3453.986788] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3454.289992] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3454.597334] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3454.886667] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3455.216567] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3455.511512] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3455.835951] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3456.145213] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3456.468992] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3456.775368] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3457.075881] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3457.389484] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3457.705728] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3458.009613] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3458.337840] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3458.641060] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3458.998112] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3459.344893] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3459.665627] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3460.010173] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3460.344067] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3460.659183] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3460.984230] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3461.317115] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3461.656134] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3461.984544] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3462.352807] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3462.656274] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3462.962687] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3463.304904] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3463.643327] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3463.978309] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3464.316515] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3464.643849] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3464.969732] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3465.271526] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3465.577864] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3465.871875] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3466.194084] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3466.492870] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3466.793542] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3467.098967] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3467.416291] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3467.713824] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3468.011494] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3468.321583] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3468.647454] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3468.948547] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3469.245914] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3469.554804] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3469.876448] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3470.206701] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3470.498579] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3470.797014] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3471.099756] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3471.423644] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3471.731465] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3472.043636] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3472.372125] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3472.712233] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3473.050414] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3473.422467] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3473.764920] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3474.091148] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3474.405659] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3474.711165] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3475.041359] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3475.363320] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3475.671777] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3475.983144] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3476.281176] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3476.592060] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3476.906331] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3477.232862] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3477.536569] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3477.832850] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3478.162961] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3478.504110] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3478.807474] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3479.117996] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3479.445149] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3479.781507] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3480.094397] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3480.395959] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3480.693069] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3481.009436] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3481.337702] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 3481.340753] Lustre: Skipped 383 previous similar messages [ 3481.347424] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3481.352498] Lustre: Skipped 383 previous similar messages [ 3481.354767] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 3481.357646] Lustre: Skipped 383 previous similar messages [ 3481.359688] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3481.364695] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 3481.367434] Lustre: Skipped 383 previous similar messages [ 3481.663548] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3481.960595] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3482.267260] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3482.553783] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3482.848335] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3483.154126] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3483.460852] LustreError: 171642:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 3483.463482] LustreError: 171642:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 3483.464805] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3483.768951] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3484.080883] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3484.412634] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3484.740553] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3485.056229] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3485.375716] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3485.699163] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3486.024687] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3486.355533] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3486.659732] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3486.992228] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3487.322549] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3487.642414] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3487.968438] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3488.306470] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3488.646505] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3488.952119] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3489.254894] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3489.580979] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3489.925359] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3490.241411] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3490.559690] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3490.918800] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3491.230953] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3491.549409] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3491.860054] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3492.190080] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3492.532149] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3492.879763] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3493.235181] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3493.588353] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3493.919548] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3494.265635] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3494.617917] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3494.953372] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3495.268059] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3495.593342] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3495.930775] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3496.243258] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3496.554619] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3496.860520] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3497.175680] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3497.515808] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3497.816895] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3498.133430] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3498.448699] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3498.777640] LustreError: 171642:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 3498.781488] LustreError: 171642:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 3498.782091] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3499.115976] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3499.464973] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3499.804657] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3500.174288] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3500.502358] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3500.849170] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3501.202112] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3501.538942] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3501.857362] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3502.203824] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3502.549641] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3502.905678] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3503.246963] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3503.575596] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3503.889169] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3504.259550] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3504.598751] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3504.921947] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3505.238514] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3505.573552] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3505.920316] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3506.264805] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3506.570097] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3506.877035] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3507.170630] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3507.469667] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3507.776328] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3508.083647] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3508.387029] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3508.697068] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3508.999257] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3509.318409] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3509.616744] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3509.926888] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3510.228055] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3510.563559] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3510.848063] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3511.138042] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3511.425243] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3511.730227] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3512.034707] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3512.342402] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3512.646082] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3512.955330] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3513.254126] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3513.564531] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3513.860533] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3514.171069] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3514.474859] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3514.779539] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3515.092851] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3515.414436] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3515.702472] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3515.990015] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3516.286050] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3516.610066] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3516.934409] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3517.288417] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3517.592377] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3517.908541] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3518.232073] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3518.536103] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3518.856792] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3519.165489] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3519.452876] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3519.747779] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3520.047054] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3520.346067] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3520.630150] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3520.922321] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3521.210061] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3521.514502] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3521.796641] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3522.083344] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3522.367367] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3522.671859] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3522.998855] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3523.303410] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3523.645528] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3523.973930] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3524.310085] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3524.621639] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3524.938350] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3525.250304] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3525.590158] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3525.933772] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3526.273824] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3526.585531] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3526.904070] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3527.234837] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3527.572126] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3527.909837] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3528.300427] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3528.651172] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3528.954313] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3529.260706] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3529.602363] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3529.899746] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3530.183337] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3530.492476] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3530.810273] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3531.155603] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3531.469855] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3531.787683] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3532.098844] LustreError: 171642:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 3532.102504] LustreError: 171642:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 3532.104530] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3532.411150] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3532.729442] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3533.036710] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3533.342723] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3533.656249] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3533.969484] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3534.287561] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3534.634673] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3534.980471] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3535.293835] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3535.649495] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3535.988821] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3536.302493] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3536.637284] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3536.942487] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3537.296153] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3537.643857] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3538.023073] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3538.376507] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3538.722586] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3539.065261] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3539.393732] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3539.769493] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3540.135616] LustreError: 171642:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 3540.140380] LustreError: 171642:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 3540.141320] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3540.503379] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3540.834491] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3541.158837] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3541.501450] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3541.858466] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3542.201587] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3542.529740] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3542.839403] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3543.158306] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3543.513646] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3543.873296] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3544.185457] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3544.524471] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3544.931320] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3545.279442] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3545.669477] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3546.039045] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3546.386713] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3546.719734] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3547.045864] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3547.396854] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3547.711867] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3548.024257] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3548.334174] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3548.664186] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3548.979785] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3549.299616] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3549.621603] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3549.957979] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3550.323270] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3550.631339] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3550.992907] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3551.313209] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3551.636914] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3551.937387] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3552.237674] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3638.410970] LustreError: 173807:0:(ofd_dev.c:1780:ofd_create_hdl()) lustre-OST0000: unable to precreate: rc = -28 [ 3638.414526] LustreError: 171642:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -28 [ 3716.067416] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3716.067782] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3716.073332] Lustre: Skipped 1 previous similar message [ 3721.184373] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3721.187037] Lustre: Skipped 1 previous similar message [ 3722.144177] Lustre: server umount lustre-OST0000 complete [ 3726.306922] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3726.307210] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3726.310870] Lustre: Skipped 1 previous similar message [ 3730.531988] Lustre: server umount lustre-MDT0000 complete [ 3731.430572] LustreError: 174288:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3731.457901] LustreError: 174288:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [ 3734.214925] LustreError: 171603:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788362695 with bad export cookie 7871621318269174012 [ 3734.218070] LustreError: MGC192.168.202.138@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3734.223167] LustreError: 171603:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 3734.475134] Lustre: server umount lustre-MDT0001 complete [ 3742.598957] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing unload_modules_local [ 3744.807705] Key type lgssc unregistered [ 3745.028891] LNet: 226655:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3745.035714] LNetError: 226655:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3745.044966] LNet: Removed LNI 192.168.202.138@tcp [ 3745.672100] Key type .llcrypt unregistered [ 3745.673096] Key type ._llcrypt unregistered [ 3766.340718] Key type ._llcrypt registered [ 3766.342918] Key type .llcrypt registered [ 3766.439371] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing load_modules_local [ 3767.220304] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3767.238097] alg: No test for adler32 (adler32-zlib) [ 3768.321186] Lustre: Lustre: Build Version: 2.17.56_1_g896c54c [ 3768.494822] LNet: Added LNI 192.168.202.138@tcp [8/256/0/180] [ 3770.151197] Key type lgssc registered [ 3771.330863] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3777.059738] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3777.069577] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3778.351357] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3781.288239] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 3787.749956] LustreError: 228052:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3787.759825] LustreError: 228052:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 3788.649255] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3789.063968] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3793.050703] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 3800.711798] Lustre: DEBUG MARKER: oleg238-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3808.222564] Lustre: DEBUG MARKER: oleg238-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3814.504760] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3814.845806] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3820.361605] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 3823.036821] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49665 to 0x280000401:49696) [ 3826.531614] Lustre: DEBUG MARKER: oleg238-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3839.462426] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 3840.386088] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3840.395131] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 3840.399839] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3840.405102] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 3840.405247] LustreError: 228071:0:(osp_precreate.c:1268:osp_precreate_thread()) cfs_fail_timeout id 2107 sleeping for 20000ms [ 3843.532336] Lustre: lustre-OST0000: Not available for connect from 192.168.202.38@tcp (stopping) [ 3844.580602] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3844.582275] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3844.583872] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3844.587847] Lustre: Skipped 1 previous similar message [ 3847.750363] Lustre: server umount lustre-OST0000 complete [ 3860.415112] LustreError: 228071:0:(osp_precreate.c:1268:osp_precreate_thread()) cfs_fail_timeout id 2107 awake [ 3878.340424] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3878.548610] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3882.224738] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 3884.708409] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3884.715790] LustreError: lustre-OST0000-osc-MDT0001: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3884.717236] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 3884.733900] Lustre: Skipped 1 previous similar message [ 3886.793498] Lustre: DEBUG MARKER: oleg238-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3887.726179] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid in FULL state after 0 sec [ 3891.085903] Lustre: DEBUG MARKER: oleg238-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff93d043eb8800.ost_server_uuid 50 [ 3892.035494] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff93d043eb8800.ost_server_uuid in FULL state after 0 sec [ 3895.268778] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3895.272205] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3895.272528] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3895.272533] Lustre: Skipped 1 previous similar message [ 3895.280993] Lustre: Skipped 1 previous similar message [ 3900.350651] Lustre: server umount lustre-OST0000 complete [ 3905.507489] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3905.509014] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3905.519590] Lustre: Skipped 1 previous similar message [ 3905.522440] Lustre: Skipped 1 previous similar message [ 3910.627885] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3910.632464] Lustre: Skipped 2 previous similar messages [ 3917.279154] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 3917.404514] Lustre: server umount lustre-MDT0000 complete [ 3919.621578] LustreError: 230803:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788362881 with bad export cookie 10178916481232211160 [ 3919.627368] LustreError: MGC192.168.202.138@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3919.630542] LustreError: 230803:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 3919.871835] Lustre: server umount lustre-MDT0001 complete [ 3925.887193] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing unload_modules_local [ 3927.557756] Key type lgssc unregistered [ 3927.776216] LNet: 233277:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3927.783149] LNetError: 233277:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3928.810207] LNet: Removed LNI 192.168.202.138@tcp [ 3929.412570] Key type .llcrypt unregistered [ 3929.413646] Key type ._llcrypt unregistered [ 3949.486659] Key type ._llcrypt registered [ 3949.489581] Key type .llcrypt registered [ 3949.597705] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing load_modules_local [ 3950.285356] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3950.322047] alg: No test for adler32 (adler32-zlib) [ 3951.315682] Lustre: Lustre: Build Version: 2.17.56_1_g896c54c [ 3951.475154] LNet: Added LNI 192.168.202.138@tcp [8/256/0/180] [ 3953.119156] Key type lgssc registered [ 3953.945202] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3960.090784] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3960.106637] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3961.369130] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3964.259332] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 3969.348614] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3969.591230] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3972.345109] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 3973.885839] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 3977.633106] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3977.899189] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3981.214600] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 3984.986964] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3985.091776] Lustre: lustre-OST0001: new disk, initializing [ 3985.094428] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 3985.100950] Lustre: 237291:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3985.151569] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 3986.218962] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49728) [ 3988.027488] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 3991.079184] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 3991.084254] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 3991.132631] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 3995.106515] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 3999.973916] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 4010.466573] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4010.469396] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4010.486316] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4011.491761] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4011.496257] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4011.500072] Lustre: Skipped 2 previous similar messages [ 4011.504044] Lustre: Skipped 3 previous similar messages [ 4014.890136] Lustre: server umount lustre-MDT0000 complete [ 4016.610398] LustreError: 234985:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4016.617157] LustreError: 234985:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 5 previous similar messages [ 4016.875815] LustreError: 238178:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788362978 with bad export cookie 17410256813023165393 [ 4016.879894] LustreError: MGC192.168.202.138@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4016.880989] LustreError: 238178:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 4016.891775] LustreError: 235720:0:(osp_object.c:618:osp_attr_get()) lustre-MDT0000-osp-MDT0001: osp_attr_get update error [0x200001b72:0x1:0x0]: rc = -5 [ 4017.046365] Lustre: server umount lustre-MDT0001 complete [ 4028.974533] Lustre: server umount lustre-OST0000 complete [ 4040.799528] Lustre: server umount lustre-OST0001 complete [ 4054.868556] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing load_modules_local [ 4061.173487] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4061.429183] LustreError: 240399:0:(ldlm_lib.c:1192:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4061.492501] Lustre: mylustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4063.802813] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 4066.784833] LustreError: 240400:0:(ldlm_lib.c:1192:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4068.424167] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4070.782072] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 4072.564154] Lustre: 241539:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 4076.894309] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4077.101390] Lustre: mylustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4077.106883] Lustre: Skipped 1 previous similar message [ 4079.137706] LustreError: 241892:0:(ldlm_lib.c:1192:target_handle_connect()) mylustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4079.147867] Lustre: mylustre-OST0000: new connection from mylustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:100 to 0x280000400:129) [ 4080.395882] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 4084.198795] LustreError: 241892:0:(ldlm_lib.c:1192:target_handle_connect()) mylustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4084.852795] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4087.022995] Lustre: mylustre-OST0000: new connection from mylustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49760) [ 4087.517672] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 4092.830957] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 4095.689722] Lustre: 243406:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 4104.619420] Lustre: server umount mylustre-MDT0000 complete [ 4104.677431] LustreError: mylustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4104.685140] Lustre: mylustre-MDT0000-osp-MDT0001: Connection to mylustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4104.696251] LustreError: 240400:0:(ldlm_lib.c:1192:target_handle_connect()) mylustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4107.757693] Lustre: mylustre-MDT0000-lwp-OST0000: Connection to mylustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4107.781337] Lustre: Skipped 2 previous similar messages [ 4107.793904] LustreError: 240374:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788363069 with bad export cookie 17410256813023168256 [ 4107.801627] LustreError: MGC192.168.202.138@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4107.804015] LustreError: 240374:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 4110.060106] Lustre: server umount mylustre-MDT0001 complete [ 4122.121558] Lustre: server umount mylustre-OST0000 complete [ 4134.073395] Lustre: server umount mylustre-OST0001 complete [ 4143.913737] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing load_modules_local [ 4148.322046] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4148.512457] LustreError: 245712:0:(ldlm_lib.c:1192:target_handle_connect()) tfs-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4148.521282] LustreError: 245712:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 4 previous similar messages [ 4148.545220] Lustre: tfs-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4148.548546] Lustre: Skipped 1 previous similar message [ 4150.614766] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 4155.743235] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4158.410171] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 4160.117337] Lustre: 246852:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 4163.995145] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4166.325961] Lustre: tfs-OST0000: new connection from tfs-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:100 to 0x280000400:161) [ 4167.397791] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 4173.223682] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4174.460294] Lustre: tfs-OST0000: new connection from tfs-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49792) [ 4174.463570] Lustre: tfs-OST0001: new connection from tfs-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:35 to 0x2c0000400:65) [ 4177.324420] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 4182.827965] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 4185.200652] Lustre: 248722:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 4190.182911] Lustre: tfs-MDT0000-lwp-OST0001: Connection to tfs-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4190.185703] Lustre: tfs-MDT0000: Not available for connect from 0@lo (stopping) [ 4190.190030] Lustre: Skipped 2 previous similar messages [ 4190.196708] Lustre: Skipped 3 previous similar messages [ 4195.302878] Lustre: tfs-MDT0000: Not available for connect from 0@lo (stopping) [ 4195.306829] Lustre: Skipped 3 previous similar messages [ 4195.454495] Lustre: server umount tfs-MDT0000 complete [ 4197.666567] LustreError: 245688:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788363159 with bad export cookie 17410256813023170209 [ 4197.669907] LustreError: MGC192.168.202.138@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4197.673021] LustreError: 245688:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 4197.888773] Lustre: server umount tfs-MDT0001 complete [ 4209.644734] Lustre: server umount tfs-OST0000 complete [ 4222.211954] Lustre: server umount tfs-OST0001 complete [ 4236.136681] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing load_modules_local [ 4242.588743] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4242.867864] LustreError: 250973:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4242.877460] LustreError: 250973:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [ 4242.931579] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4242.934865] Lustre: Skipped 3 previous similar messages [ 4246.233241] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 4252.302385] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4255.400957] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 4257.196672] Lustre: 252113:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 4261.568775] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4262.833503] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:100 to 0x280000400:193) [ 4265.829585] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 4269.063963] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49824) [ 4271.414784] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4275.861492] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 4276.724092] Lustre: lustre-OST0001: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:35 to 0x2c0000400:97) [ 4281.106719] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 4283.612407] Lustre: 253983:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 4287.829185] Lustre: DEBUG MARKER: == conf-sanity test 104a: Make sure user defined options are reflected in mount ========================================================== 11:34:07 (1788363247) [ 4302.314563] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4302.327399] Lustre: Skipped 3 previous similar messages [ 4302.328527] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4302.337216] Lustre: Skipped 3 previous similar messages [ 4307.707976] Lustre: server umount lustre-MDT0000 complete [ 4310.115418] LustreError: 253985:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788363271 with bad export cookie 17410256813023172127 [ 4310.122680] LustreError: MGC192.168.202.138@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4310.124223] LustreError: 253985:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4312.544390] LustreError: 254352:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4312.548836] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 4312.556026] LustreError: 254352:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 6 previous similar messages [ 4312.562352] Lustre: Skipped 5 previous similar messages [ 4316.421876] Lustre: server umount lustre-MDT0001 complete [ 4324.638825] Lustre: server umount lustre-OST0000 complete [ 4341.215163] Lustre: lustre-OST0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 4341.314553] Lustre: server umount lustre-OST0001 complete [ 4344.918490] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_hostid [ 4350.011984] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing load_modules_local [ 4370.249782] LDISKFS-fs (dm-0): Mount option "noacl" will be removed by 3.5 [ 4370.249782] Contact linux-ldiskfs@vger.kernel.org if you think we should keep it. [ 4370.249782] [ 4370.266241] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: acl,user_xattr,errors=remount-ro,noacl,no_mbcache,nodelalloc [ 4370.478131] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 4370.494411] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 4370.584845] Lustre: lustre-MDT0000: new disk, initializing [ 4370.616850] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4370.619823] Lustre: Skipped 3 previous similar messages [ 4370.627337] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 4372.999299] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 4379.776714] LDISKFS-fs (dm-1): Mount option "noacl" will be removed by 3.5 [ 4379.776714] Contact linux-ldiskfs@vger.kernel.org if you think we should keep it. [ 4379.776714] [ 4379.790092] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: acl,user_xattr,errors=remount-ro,noacl,no_mbcache,nodelalloc [ 4379.870543] Lustre: 257164:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 4379.925165] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 4379.928079] Lustre: Skipped 1 previous similar message [ 4379.989419] Lustre: lustre-MDT0001: new disk, initializing [ 4380.088565] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 4380.095984] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 4382.769121] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 4389.526497] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4389.728221] Lustre: lustre-OST0000: new disk, initializing [ 4389.731219] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 4389.737281] Lustre: 258743:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 4391.526475] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 4391.531711] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 4391.608466] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 4393.560649] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 4400.463828] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4400.563902] Lustre: lustre-OST0001: new disk, initializing [ 4400.566447] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 4400.569836] Lustre: 259618:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 4402.604791] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 4402.609687] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 4402.664598] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 4404.190120] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 4411.685323] Lustre: DEBUG MARKER: == conf-sanity test 104b: Mount uses last flock argument ========================================================== 11:36:11 (1788363371) [ 4418.707721] Lustre: DEBUG MARKER: == conf-sanity test 105: check file creation for ro and rw bind mnt pt ========================================================== 11:36:18 (1788363378) [ 4432.866297] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4432.867850] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4432.874243] LustreError: Skipped 1 previous similar message [ 4432.884662] Lustre: Skipped 2 previous similar messages [ 4432.893502] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4432.898444] Lustre: Skipped 3 previous similar messages [ 4437.470166] Lustre: server umount lustre-OST0000 complete [ 4441.568602] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4443.622201] LustreError: 258726:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4443.630843] LustreError: 258726:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 4 previous similar messages [ 4445.910370] Lustre: server umount lustre-MDT0000 complete [ 4448.293899] LustreError: 259617:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788363409 with bad export cookie 17410256813023173905 [ 4448.301668] LustreError: 259617:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 4448.302761] LustreError: MGC192.168.202.138@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4454.671244] Lustre: server umount lustre-MDT0001 complete [ 4461.540398] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing unload_modules_local [ 4474.605681] Lustre: server umount lustre-OST0001 complete [ 4478.443204] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_hostid [ 4489.896685] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing load_modules_local [ 4513.600041] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4513.751683] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 4513.772220] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 4513.827700] Lustre: lustre-MDT0000: new disk, initializing [ 4513.864169] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4513.866248] Lustre: Skipped 3 previous similar messages [ 4513.875488] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 4516.586983] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 4523.723946] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4523.806958] Lustre: 264149:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 4523.995698] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 4526.590895] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 4533.108259] Lustre: DEBUG MARKER: oleg238-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4536.182491] Lustre: DEBUG MARKER: oleg238-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4539.826485] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4539.967224] Lustre: lustre-OST0000: new disk, initializing [ 4539.969677] Lustre: Skipped 1 previous similar message [ 4539.972673] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 4539.975707] Lustre: Skipped 2 previous similar messages [ 4539.979159] Lustre: 266114:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 4541.725721] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 4541.729481] Lustre: Skipped 1 previous similar message [ 4541.731409] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 4541.792597] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 4543.509483] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 4549.132255] Lustre: DEBUG MARKER: oleg238-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4557.284180] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4557.284830] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4557.287398] LustreError: Skipped 1 previous similar message [ 4557.288747] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4557.297138] Lustre: Skipped 6 previous similar messages [ 4557.307421] Lustre: Skipped 9 previous similar messages [ 4558.513826] Lustre: server umount lustre-OST0000 complete [ 4566.645834] Lustre: server umount lustre-MDT0000 complete [ 4568.823383] LustreError: 264140:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788363530 with bad export cookie 17410256813023175368 [ 4568.826662] LustreError: MGC192.168.202.138@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4568.829541] LustreError: 264140:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4568.993288] Lustre: server umount lustre-MDT0001 complete [ 4574.176302] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing unload_modules_local [ 4575.459196] Key type lgssc unregistered [ 4575.624051] LNet: 267871:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4575.627336] LNetError: 267871:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4575.642479] LNet: Removed LNI 192.168.202.138@tcp [ 4576.070245] Key type .llcrypt unregistered [ 4576.072126] Key type ._llcrypt unregistered [ 4589.845429] Key type ._llcrypt registered [ 4589.846753] Key type .llcrypt registered [ 4589.896515] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing load_modules_local [ 4590.344915] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4590.376991] alg: No test for adler32 (adler32-zlib) [ 4591.258625] Lustre: Lustre: Build Version: 2.17.56_1_g896c54c [ 4591.367481] LNet: Added LNI 192.168.202.138@tcp [8/256/0/180] [ 4592.967372] Key type lgssc registered [ 4593.451900] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4596.798091] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4596.809136] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4598.034305] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4599.888192] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 4604.019182] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4604.234072] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4606.275424] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 4609.645492] Lustre: DEBUG MARKER: oleg238-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4612.220791] Lustre: DEBUG MARKER: oleg238-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4615.486890] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4615.661521] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4618.578483] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 4621.869913] Lustre: DEBUG MARKER: oleg238-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4622.386473] Lustre: Setting parameter lustre-OST0000.ost.unknown_param=50 in log lustre-OST0000 [ 4622.835015] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:3 to 0x280000401:33) [ 4627.938733] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4627.942924] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4627.959823] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4629.030062] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4629.637516] Lustre: server umount lustre-OST0000 complete [ 4634.083720] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4634.085117] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4634.100214] Lustre: Skipped 1 previous similar message [ 4634.103690] Lustre: Skipped 1 previous similar message [ 4639.201605] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4639.204586] Lustre: Skipped 2 previous similar messages [ 4644.320682] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4644.324446] Lustre: Skipped 1 previous similar message [ 4646.367108] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 4646.457494] Lustre: server umount lustre-MDT0000 complete [ 4648.348560] LustreError: 271152:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788363609 with bad export cookie 3871696168249101171 [ 4648.350977] LustreError: MGC192.168.202.138@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4648.354561] LustreError: 271152:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4648.513903] Lustre: server umount lustre-MDT0001 complete [ 4653.498459] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing unload_modules_local [ 4654.826117] Key type lgssc unregistered [ 4654.980173] LNet: 272891:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4654.987911] LNetError: 272891:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4655.005181] LNet: Removed LNI 192.168.202.138@tcp [ 4655.455212] Key type .llcrypt unregistered [ 4655.456763] Key type ._llcrypt unregistered [ 4664.751629] Key type ._llcrypt registered [ 4664.753356] Key type .llcrypt registered [ 4664.796899] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing load_modules_local [ 4665.197260] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4665.248089] alg: No test for adler32 (adler32-zlib) [ 4666.139664] Lustre: Lustre: Build Version: 2.17.56_1_g896c54c [ 4666.247612] LNet: Added LNI 192.168.202.138@tcp [8/256/0/180] [ 4667.839165] Key type lgssc registered [ 4668.365405] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4672.715872] Lustre: lustre-OST0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4672.726909] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4688.160370] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 4693.279218] LustreError: 274078:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.202.138@tcp: failed processing log, type 4: rc = -110 [ 4719.907717] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4722.965163] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 4727.092255] Lustre: DEBUG MARKER: oleg238-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4728.371204] Lustre: server umount lustre-OST0000 complete [ 4737.522130] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing unload_modules_local [ 4739.118969] Key type lgssc unregistered [ 4739.304602] LNet: 275705:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4739.308914] LNetError: 275705:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4739.325487] LNet: Removed LNI 192.168.202.138@tcp [ 4739.755412] Key type .llcrypt unregistered [ 4739.756615] Key type ._llcrypt unregistered [ 4760.568461] Key type ._llcrypt registered [ 4760.569667] Key type .llcrypt registered [ 4760.631456] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing load_modules_local [ 4761.052353] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4761.060788] alg: No test for adler32 (adler32-zlib) [ 4761.957792] Lustre: Lustre: Build Version: 2.17.56_1_g896c54c [ 4762.067605] LNet: Added LNI 192.168.202.138@tcp [8/256/0/180] [ 4763.663151] Key type lgssc registered [ 4764.209603] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4771.110809] loop0: detected capacity change from 0 to 204800000 [ 4771.233982] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 4771.239471] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 4771.392472] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 4771.997458] loop0: detected capacity change from 0 to 204800000 [ 4772.093429] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 4772.102481] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 4772.283448] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 4772.894176] loop0: detected capacity change from 0 to 204800000 [ 4772.982808] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 4772.991583] blk_update_request: operation not supported error, dev loop0, sector 8224 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 4773.049401] blk_update_request: operation not supported error, dev loop0, sector 58264 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 4773.658702] loop0: detected capacity change from 0 to 204800000 [ 4773.790734] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 4774.350723] loop0: detected capacity change from 0 to 204800000 [ 4774.368300] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 4776.320409] loop0: detected capacity change from 0 to 204800000 [ 4776.335694] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 4778.092167] loop0: detected capacity change from 0 to 204800000 [ 4778.107258] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 4779.877301] loop0: detected capacity change from 0 to 204800000 [ 4779.892730] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 4781.629351] loop0: detected capacity change from 0 to 204800000 [ 4781.670596] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4781.681805] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4781.722208] Lustre: 279092:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 4781.727377] Lustre: 279092:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index 'nodemap' with [0x200000003:0x3:0x0]: rc = 0 [ 4781.732431] Lustre: 279092:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000003:0xa:0x0]: rc = 0 [ 4781.737840] Lustre: 279092:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000-MDT0000' with [0x200000005:0x3:0x0]: rc = 0 [ 4781.741839] Lustre: 279092:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000003:0xc:0x0]: rc = 0 [ 4781.746446] Lustre: 279092:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000-MDT0000' with [0x200000005:0x2:0x0]: rc = 0 [ 4781.750871] Lustre: 279092:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000-MDT0000' with [0x200000005:0x1:0x0]: rc = 0 [ 4781.755423] Lustre: 279092:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000003:0xb:0x0]: rc = 0 [ 4781.761557] Lustre: 279092:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x20000' with [0x200000006:0x20000:0x0]: rc = 0 [ 4781.765848] Lustre: 279092:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1020000' with [0x200000006:0x1020000:0x0]: rc = 0 [ 4781.769843] Lustre: 279092:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2020000' with [0x200000006:0x2020000:0x0]: rc = 0 [ 4781.774957] Lustre: 279092:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000006:0x10000:0x0]: rc = 0 [ 4781.778739] Lustre: 279092:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000006:0x2010000:0x0]: rc = 0 [ 4781.783600] Lustre: 279092:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000006:0x1010000:0x0]: rc = 0 [ 4784.885913] Lustre: server umount MGS complete [ 4785.375910] loop0: detected capacity change from 0 to 204800000 [ 4785.450085] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4785.674454] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4785.699194] Lustre: lustre-MDT0000: Aborting client recovery [ 4786.130474] loop1: detected capacity change from 0 to 204800000 [ 4786.190860] LDISKFS-fs (loop1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4786.224893] Lustre: 279522:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 4786.228953] Lustre: 279522:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x10000-MDT0001' with [0x200000005:0x1:0x0]: rc = 0 [ 4786.234335] Lustre: 279522:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x1010000-MDT0001' with [0x200000005:0x2:0x0]: rc = 0 [ 4786.240063] Lustre: 279522:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x1010000' with [0x200000003:0x5:0x0]: rc = 0 [ 4786.246122] Lustre: 279522:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x2010000-MDT0001' with [0x200000005:0x3:0x0]: rc = 0 [ 4786.250052] Lustre: 279522:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x10000' with [0x200000003:0x4:0x0]: rc = 0 [ 4786.254603] Lustre: 279522:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x2010000' with [0x200000003:0x6:0x0]: rc = 0 [ 4786.267350] Lustre: Found index 1 for lustre-MDT0001, updating log [ 4786.355264] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4786.384048] Lustre: lustre-MDT0001: Aborting client recovery [ 4786.837214] loop2: detected capacity change from 0 to 204800000 [ 4786.887796] LDISKFS-fs (loop2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4786.924908] Lustre: 279602:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 4786.931911] Lustre: 279602:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 4786.939026] Lustre: 279602:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x1020000-OST0000' with [0x200000005:0x2:0x0]: rc = 0 [ 4786.946546] Lustre: 279602:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x2020000-OST0000' with [0x200000005:0x3:0x0]: rc = 0 [ 4786.951544] Lustre: 279602:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x20000-OST0000' with [0x200000005:0x1:0x0]: rc = 0 [ 4786.958327] Lustre: 279602:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 4786.963237] Lustre: 279602:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 4787.033602] Lustre: Found index 0 for lustre-OST0000, updating log [ 4787.063542] Lustre: 279621:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 4787.119050] Lustre: lustre-OST0000: Aborting recovery [ 4787.585126] loop3: detected capacity change from 0 to 204800000 [ 4787.624462] LDISKFS-fs (loop3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4787.663635] Lustre: 279686:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 4787.668940] Lustre: 279686:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x20000-OST0001' with [0x200000005:0x1:0x0]: rc = 0 [ 4787.676205] Lustre: 279686:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x2020000-OST0001' with [0x200000005:0x3:0x0]: rc = 0 [ 4787.680769] Lustre: 279686:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 4787.688326] Lustre: 279686:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x1020000-OST0001' with [0x200000005:0x2:0x0]: rc = 0 [ 4787.696077] Lustre: 279686:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 4787.702560] Lustre: 279686:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 4787.724593] Lustre: 279692:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 4787.748386] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 4787.753517] Lustre: Skipped 1 previous similar message [ 4787.768803] Lustre: lustre-OST0001: Aborting recovery [ 4792.815744] LustreError: 279466:0:(lod_dev.c:511:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 7, retries 0, failed: rc = -108 [ 4795.875204] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 4795.875789] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 4795.888691] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:5 to 0x280000400:33) [ 4795.891179] Lustre: lustre-OST0001: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:5 to 0x2c0000400:33) [ 4795.891301] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000bd0 [ 4795.900343] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000bd0 [ 4798.879211] LustreError: 279433:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0000: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0000 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0000/identity_upcall, time 259us: rc = -2 [ 4811.231799] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4811.232314] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4811.236789] Lustre: Skipped 2 previous similar messages [ 4811.239122] Lustre: Skipped 1 previous similar message [ 4816.352296] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4816.354779] Lustre: Skipped 5 previous similar messages [ 4816.643286] Lustre: server umount lustre-MDT0000 complete [ 4817.018676] LustreError: 279417:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788363778 with bad export cookie 5815893951223395477 [ 4817.020744] LustreError: MGC192.168.202.138@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4817.024241] LustreError: 279417:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 4821.472953] LustreError: 279431:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4821.473135] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4821.482589] LustreError: 279431:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [ 4821.487125] Lustre: Skipped 1 previous similar message [ 4821.492457] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 4821.494673] Lustre: Skipped 1 previous similar message [ 4822.495909] LustreError: 279431:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4822.507944] LustreError: 279431:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 4823.251261] Lustre: server umount lustre-MDT0001 complete [ 4829.774893] Lustre: server umount lustre-OST0000 complete [ 4836.329257] Lustre: server umount lustre-OST0001 complete [ 4840.336525] Lustre: DEBUG MARKER: == conf-sanity test 109a: test lctl clear_conf fsname ==== 11:43:20 (1788363800) [ 4854.246647] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_hostid [ 4859.763228] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing load_modules_local [ 4881.480148] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4881.611158] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 4881.628525] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 4881.691247] Lustre: lustre-MDT0000: new disk, initializing [ 4881.731450] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4881.747315] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 4884.352655] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 4890.894252] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4890.958878] Lustre: 283756:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 4890.988893] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 4890.992375] Lustre: Skipped 1 previous similar message [ 4891.032432] Lustre: lustre-MDT0001: new disk, initializing [ 4891.090250] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4891.121733] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 4891.128187] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 4893.751792] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 4899.461582] Lustre: DEBUG MARKER: oleg238-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4902.168892] Lustre: DEBUG MARKER: oleg238-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4906.272684] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4906.424479] Lustre: lustre-OST0000: new disk, initializing [ 4906.427076] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 4906.430209] Lustre: 285724:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 4906.506559] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4910.146944] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 4911.611894] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 4911.619930] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 4911.657385] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 4916.285246] Lustre: DEBUG MARKER: oleg238-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4919.534633] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [ 4928.698790] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [ 4937.031806] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [ 4947.118972] Lustre: Modifying parameter lustre-client.llite.max_read_ahead_mb=64 in log lustre-client [ 4983.266717] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4983.267660] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4983.270862] LustreError: Skipped 1 previous similar message [ 4983.275658] Lustre: Skipped 1 previous similar message [ 4983.276550] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4983.283284] Lustre: Skipped 3 previous similar messages [ 4988.180428] Lustre: server umount lustre-OST0000 complete [ 4993.504863] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4993.508417] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4993.511513] Lustre: Skipped 2 previous similar messages [ 4993.511774] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4993.521061] Lustre: Skipped 1 previous similar message [ 4996.204202] Lustre: server umount lustre-MDT0000 complete [ 4998.087649] LustreError: 287364:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788363959 with bad export cookie 5815893951223398186 [ 4998.088329] LustreError: MGC192.168.202.138@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4998.096914] LustreError: 287364:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 4998.327672] Lustre: server umount lustre-MDT0001 complete [ 5003.281260] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5005.191478] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 5007.500151] Lustre: server umount MGS complete [ 5013.625466] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5013.902060] LustreError: 289790:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5013.913788] LustreError: 289803:0:(qmt_pool.c:1645:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [ 5013.953219] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5016.402870] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 5019.105845] LustreError: 289791:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5021.075601] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5023.476047] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 5027.683956] Lustre: DEBUG MARKER: oleg238-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 5030.684094] Lustre: DEBUG MARKER: oleg238-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 5034.426626] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5037.961947] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 5042.144320] Lustre: DEBUG MARKER: oleg238-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 5064.673810] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 5064.676537] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5064.681653] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5066.723673] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 5067.807913] Lustre: server umount lustre-OST0000 complete [ 5075.737827] Lustre: server umount lustre-MDT0000 complete [ 5076.960334] LustreError: 289791:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5076.964746] LustreError: 289791:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 5077.264407] LustreError: 289771:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788364038 with bad export cookie 5815893951223400083 [ 5077.264699] LustreError: MGC192.168.202.138@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5077.271880] LustreError: 289771:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 5077.450159] Lustre: server umount lustre-MDT0001 complete [ 5082.058496] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing unload_modules_local [ 5083.408344] Key type lgssc unregistered [ 5083.562421] LNet: 293729:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5083.567298] LNetError: 293729:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5083.577389] LNet: Removed LNI 192.168.202.138@tcp [ 5083.948550] Key type .llcrypt unregistered [ 5083.949612] Key type ._llcrypt unregistered [ 5099.892984] Key type ._llcrypt registered [ 5099.894132] Key type .llcrypt registered [ 5099.935852] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_hostid [ 5107.083517] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing load_modules_local [ 5107.577636] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5107.600025] alg: No test for adler32 (adler32-zlib) [ 5108.480905] Lustre: Lustre: Build Version: 2.17.56_1_g896c54c [ 5108.590113] LNet: Added LNI 192.168.202.138@tcp [8/256/0/180] [ 5110.183155] Key type lgssc registered [ 5110.685865] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5125.220134] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5125.232391] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5126.357086] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 5126.369356] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 5126.404883] Lustre: lustre-MDT0000: new disk, initializing [ 5126.440821] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5126.452506] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 5128.324752] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 5133.779702] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5133.848618] Lustre: 297659:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 5133.895817] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 5133.899270] Lustre: Skipped 1 previous similar message [ 5133.942552] Lustre: lustre-MDT0001: new disk, initializing [ 5134.001259] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5134.029162] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 5134.033622] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 5135.855171] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 5140.667830] Lustre: DEBUG MARKER: oleg238-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 5143.379752] Lustre: DEBUG MARKER: oleg238-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 5146.558524] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5146.701395] Lustre: lustre-OST0000: new disk, initializing [ 5146.703652] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 5146.707051] Lustre: 299626:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 5146.748510] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5149.384593] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 5154.181558] Lustre: DEBUG MARKER: oleg238-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 5154.314836] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 5154.319447] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 5154.380051] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 5158.033845] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [ 5163.840803] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [ 5172.015511] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [ 5179.965416] Lustre: Modifying parameter lustre-client.llite.max_read_ahead_mb=64 in log lustre-client [ 5225.955710] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 5225.958315] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5225.963419] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5225.965777] Lustre: Skipped 1 previous similar message [ 5227.543307] Lustre: server umount lustre-OST0000 complete [ 5231.072738] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5231.073474] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5231.080380] Lustre: Skipped 2 previous similar messages [ 5236.192771] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5236.197654] Lustre: Skipped 2 previous similar messages [ 5241.313161] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5241.316368] Lustre: Skipped 1 previous similar message [ 5243.359186] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 5243.443784] Lustre: server umount lustre-MDT0000 complete [ 5244.932212] LustreError: 299625:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788364206 with bad export cookie 7807964010416516938 [ 5244.936015] LustreError: MGC192.168.202.138@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5244.936196] LustreError: 299625:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 5245.080683] Lustre: server umount lustre-MDT0001 complete [ 5248.595445] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5250.287278] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 5252.102968] Lustre: server umount MGS complete [ 5256.562000] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5256.759090] LustreError: 303602:0:(qmt_pool.c:1645:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [ 5256.782726] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5258.275082] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 5261.593037] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5261.735988] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5263.181296] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 5265.891117] Lustre: DEBUG MARKER: oleg238-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 5267.974153] Lustre: DEBUG MARKER: oleg238-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 5270.688613] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5270.816519] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5273.203518] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 5275.899556] Lustre: DEBUG MARKER: oleg238-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 5312.995047] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 5312.995131] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5312.998393] LustreError: Skipped 2 previous similar messages [ 5312.999079] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5313.002227] Lustre: Skipped 1 previous similar message [ 5313.006892] Lustre: Skipped 1 previous similar message [ 5315.858835] Lustre: server umount lustre-OST0000 complete [ 5318.112440] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5318.112860] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 5318.118267] Lustre: Skipped 1 previous similar message [ 5323.232099] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5323.235273] Lustre: Skipped 3 previous similar messages [ 5323.293434] Lustre: server umount lustre-MDT0000 complete [ 5324.593333] LustreError: 306334:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788364286 with bad export cookie 7807964010416518884 [ 5324.596979] LustreError: MGC192.168.202.138@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5324.597475] LustreError: 306334:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 5324.717621] Lustre: server umount lustre-MDT0001 complete [ 5328.417987] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing unload_modules_local [ 5329.459589] Key type lgssc unregistered [ 5329.602434] LNet: 307553:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5329.605524] LNetError: 307553:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5329.612299] LNet: Removed LNI 192.168.202.138@tcp [ 5329.941472] Key type .llcrypt unregistered [ 5329.942882] Key type ._llcrypt unregistered [ 5336.893567] Key type ._llcrypt registered [ 5336.894929] Key type .llcrypt registered [ 5336.934447] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_hostid [ 5343.128308] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing load_modules_local [ 5343.556185] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5343.583872] alg: No test for adler32 (adler32-zlib) [ 5344.444387] Lustre: Lustre: Build Version: 2.17.56_1_g896c54c [ 5344.537452] LNet: Added LNI 192.168.202.138@tcp [8/256/0/180] [ 5346.119127] Key type lgssc registered [ 5346.508293] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5354.946024] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing load_modules_local [ 5360.246745] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5360.257087] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5361.344442] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 5361.354270] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 5361.390504] Lustre: lustre-MDT0000: new disk, initializing [ 5361.418440] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5361.425671] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 5362.873970] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 5367.240525] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5367.343843] Lustre: lustre-OST0000: new disk, initializing [ 5367.346134] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 5367.349576] Lustre: Skipped 1 previous similar message [ 5367.353435] Lustre: 311573:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 5367.385664] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5369.391102] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 5370.305533] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 5370.309530] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 5370.320388] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 5564.895892] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5564.900573] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5566.622619] Lustre: server umount lustre-MDT0000 complete [ 5567.990312] LustreError: 312454:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788364529 with bad export cookie 2932752127802110356 [ 5567.992526] LustreError: MGC192.168.202.138@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5567.993693] LustreError: 312454:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 5574.101400] Lustre: server umount lustre-OST0000 complete [ 5582.442547] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_hostid [ 5585.776693] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing load_modules_local [ 5598.180204] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5598.257134] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 5598.267719] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 5598.301758] Lustre: lustre-MDT0000: new disk, initializing [ 5598.324202] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5598.331201] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 5599.751831] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 5603.834863] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5603.875366] Lustre: 315819:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 5603.895397] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 5603.897830] Lustre: Skipped 1 previous similar message [ 5603.930296] Lustre: lustre-MDT0001: new disk, initializing [ 5603.963838] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5603.977372] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 5603.980619] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 5605.294492] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 5608.961569] Lustre: DEBUG MARKER: oleg238-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 5610.688752] Lustre: DEBUG MARKER: oleg238-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 5612.832200] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5612.927138] Lustre: lustre-OST0000: new disk, initializing [ 5612.928435] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 5612.930332] Lustre: 317785:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 5612.956163] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5614.771048] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 5618.362157] Lustre: DEBUG MARKER: oleg238-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 5619.716846] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 5621.752538] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 5621.756752] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 5621.790216] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 5622.837117] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 3 sec [ 5624.127417] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 5624.196917] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 5626.849152] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 5626.851628] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5626.856072] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5626.857529] Lustre: Skipped 1 previous similar message [ 5630.928679] Lustre: server umount lustre-OST0000 complete [ 5635.039791] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 5635.041923] LustreError: Skipped 1 previous similar message [ 5635.043156] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5635.046545] Lustre: Skipped 1 previous similar message [ 5635.048323] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5637.089213] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5637.089345] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5637.097511] Lustre: Skipped 1 previous similar message [ 5638.290360] Lustre: server umount lustre-MDT0000 complete [ 5639.583072] LustreError: 315810:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788364601 with bad export cookie 2932752127805051133 [ 5639.585825] LustreError: MGC192.168.202.138@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5639.586257] LustreError: 315810:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 5639.695223] Lustre: server umount lustre-MDT0001 complete [ 5640.896417] Lustre: DEBUG MARKER: SKIP: conf-sanity test_111 skipping SLOW test 111 [ 5641.509680] Lustre: DEBUG MARKER: == conf-sanity test 112a: mount OST with no_create option ========================================================== 11:56:41 (1788364601) [ 5643.761782] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5643.942723] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5645.294262] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 5648.272834] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5649.895728] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 5652.262621] Lustre: DEBUG MARKER: oleg238-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 5654.108287] Lustre: DEBUG MARKER: oleg238-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 5656.484402] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5658.558781] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 5661.137263] Lustre: DEBUG MARKER: oleg238-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 5663.611068] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5663.678965] Lustre: lustre-OST0001: new disk, initializing [ 5663.680696] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 5663.684127] Lustre: 322949:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 5663.718615] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 5663.721346] Lustre: Skipped 2 previous similar messages [ 5665.863689] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 5666.049398] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 5666.054112] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 5666.075849] LustreError: 323328:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [ 5667.106492] LustreError: 323328:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [ 5667.111020] LustreError: 323328:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 2 previous similar messages [ 5668.132527] LustreError: 323334:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [ 5668.136391] LustreError: 323334:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 1 previous similar message [ 5669.848671] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 5670.182331] LustreError: 323328:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [ 5670.187968] LustreError: 323328:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 3 previous similar messages [ 5670.960593] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 1 sec [ 5672.921146] Lustre: DEBUG MARKER: oleg238-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff93d0431a7000.ost_server_uuid 50 [ 5673.494216] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff93d0431a7000.ost_server_uuid in FULL state after 0 sec [ 5674.275759] LustreError: 323334:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [ 5674.279780] LustreError: 323334:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 7 previous similar messages [ 5675.453937] Lustre: DEBUG MARKER: oleg238-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff93d0431a7000.ost_server_uuid 50 [ 5676.053984] Lustre: DEBUG MARKER: osc.lustre-OST0001-osc-ffff93d0431a7000.ost_server_uuid in FULL state after 0 sec [ 5682.660867] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 5688.751308] Lustre: lustre-OST0001: Not available for connect from 192.168.202.38@tcp (stopping) [ 5690.851600] Lustre: lustre-OST0001-osc-MDT0001: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5690.856198] Lustre: Skipped 1 previous similar message [ 5698.984585] Lustre: lustre-OST0001: Not available for connect from 192.168.202.38@tcp (stopping) [ 5698.987370] Lustre: Skipped 5 previous similar messages [ 5702.111120] Lustre: lustre-OST0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 5702.151093] Lustre: server umount lustre-OST0001 complete [ 5706.210493] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5706.210594] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 5706.211206] LustreError: 321936:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5706.211217] LustreError: 321936:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 5709.771521] Lustre: server umount lustre-OST0000 complete [ 5716.448199] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5716.450941] Lustre: Skipped 7 previous similar messages [ 5717.211830] Lustre: server umount lustre-MDT0000 complete [ 5718.477500] LustreError: 320099:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788364680 with bad export cookie 2932752127805052001 [ 5718.480903] LustreError: MGC192.168.202.138@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5718.481655] LustreError: 320099:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 5718.603354] Lustre: server umount lustre-MDT0001 complete [ 5722.077934] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing unload_modules_local [ 5723.103538] Key type lgssc unregistered [ 5723.234356] LNet: 325640:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5723.237093] LNetError: 325640:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5723.244285] LNet: Removed LNI 192.168.202.138@tcp [ 5723.590993] Key type .llcrypt unregistered [ 5723.592269] Key type ._llcrypt unregistered [ 5732.885597] Key type ._llcrypt registered [ 5732.887018] Key type .llcrypt registered [ 5732.926829] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing load_modules_local [ 5733.391104] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5733.418673] alg: No test for adler32 (adler32-zlib) [ 5734.281423] Lustre: Lustre: Build Version: 2.17.56_1_g896c54c [ 5734.374016] LNet: Added LNI 192.168.202.138@tcp [8/256/0/180] [ 5735.959102] Key type lgssc registered [ 5736.289390] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5738.726936] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5738.732149] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5739.870809] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5741.197384] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 5743.460485] Lustre: DEBUG MARKER: oleg238-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 5745.648173] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5745.767838] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5747.073902] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 5750.208506] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5750.317525] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5752.228128] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 5754.855343] Lustre: DEBUG MARKER: oleg238-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 5756.452128] LustreError: 328717:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5756.459109] LustreError: 328717:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 5757.287260] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5757.361593] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 5759.389526] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 5761.799338] Lustre: DEBUG MARKER: oleg238-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 5764.581765] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:65) [ 5764.583347] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:33) [ 5768.637981] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing wait_import_state (FULL|IDLE) os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [ 5768.710756] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 5846.499447] Lustre: lustre-OST0001-osc-MDT0001: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5846.505379] LustreError: lustre-OST0001-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 5846.505442] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 5846.511069] Lustre: Skipped 1 previous similar message [ 5851.620687] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 5851.623258] Lustre: Skipped 1 previous similar message [ 5851.714932] Lustre: server umount lustre-OST0001 complete [ 5853.663809] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 5853.666167] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5853.670130] Lustre: Skipped 1 previous similar message [ 5853.672187] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5856.737264] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 5856.737278] LustreError: 328719:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5856.739873] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5856.740168] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5856.745383] LustreError: 328719:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 5859.548579] Lustre: server umount lustre-OST0000 complete [ 5861.856363] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5861.857554] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5861.861602] Lustre: Skipped 1 previous similar message [ 5861.863310] Lustre: Skipped 1 previous similar message [ 5867.100995] Lustre: server umount lustre-MDT0000 complete [ 5868.323146] LustreError: 327020:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788364829 with bad export cookie 5648834219121230938 [ 5868.327044] LustreError: 327020:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 5868.328405] LustreError: MGC192.168.202.138@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5868.459069] Lustre: server umount lustre-MDT0001 complete [ 5872.224746] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing unload_modules_local [ 5873.261134] Key type lgssc unregistered [ 5873.397518] LNet: 331970:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5873.399730] LNetError: 331970:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5873.407230] LNet: Removed LNI 192.168.202.138@tcp [ 5873.724052] Key type .llcrypt unregistered [ 5873.724951] Key type ._llcrypt unregistered [ 5890.713875] Key type ._llcrypt registered [ 5890.715054] Key type .llcrypt registered [ 5890.750988] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing load_modules_local [ 5891.196303] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5891.207141] alg: No test for adler32 (adler32-zlib) [ 5892.055695] Lustre: Lustre: Build Version: 2.17.56_1_g896c54c [ 5892.140318] LNet: Added LNI 192.168.202.138@tcp [8/256/0/180] [ 5893.719166] Key type lgssc registered [ 5894.065093] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5897.067508] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5897.074366] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5898.218246] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5899.469429] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 5902.111130] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5902.237205] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5903.526526] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 5904.489729] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 5906.543989] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5906.651825] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5907.682325] LustreError: 335733:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5907.685018] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:97) [ 5907.687521] LustreError: 335733:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 5908.527233] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 5911.310791] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5911.389874] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 5912.426245] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:65) [ 5913.338191] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 5917.575776] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 5921.347691] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 5927.907391] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5927.908586] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 5927.909214] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5927.911060] Lustre: Skipped 2 previous similar messages [ 5930.975064] Lustre: server umount lustre-MDT0000 complete [ 5932.273470] LustreError: 334217:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788364893 with bad export cookie 12488670053779717030 [ 5932.276458] LustreError: MGC192.168.202.138@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5932.277571] LustreError: 334217:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 5932.396586] Lustre: server umount lustre-MDT0001 complete [ 5943.877013] Lustre: server umount lustre-OST0000 complete [ 5955.369054] Lustre: server umount lustre-OST0001 complete [ 5961.385894] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing load_modules_local [ 5964.928347] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5965.065340] LustreError: 339506:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5965.084661] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5966.440861] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 5969.317715] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5970.753987] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 5971.722694] Lustre: 340646:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 5973.832158] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5973.947451] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5973.949405] Lustre: Skipped 1 previous similar message [ 5975.011181] LustreError: 341000:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5975.015452] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:129) [ 5975.805724] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 5978.573639] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5979.691826] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:97) [ 5980.520364] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 5983.645739] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 5984.853923] Lustre: 342514:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 5989.858144] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5989.861114] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5989.861929] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 5989.862961] Lustre: Skipped 3 previous similar messages [ 5989.867506] Lustre: Skipped 6 previous similar messages [ 5991.897252] Lustre: server umount lustre-MDT0000 complete [ 5993.062205] LustreError: 339486:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788364954 with bad export cookie 12488670053779718689 [ 5993.065647] LustreError: 339486:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 5993.066222] LustreError: MGC192.168.202.138@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5993.173567] Lustre: server umount lustre-MDT0001 complete [ 6004.160957] Lustre: server umount lustre-OST0000 complete [ 6014.621015] Lustre: server umount lustre-OST0001 complete [ 6015.734214] Lustre: DEBUG MARKER: SKIP: conf-sanity test_114 skipping SLOW test 114 [ 6016.206508] Lustre: DEBUG MARKER: SKIP: conf-sanity test_115 skipping excluded test 115 [ 6016.688505] Lustre: DEBUG MARKER: == conf-sanity test 116: big size MDT support ============ 12:02:57 (1788364977) [ 6023.454671] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing load_modules_local [ 6026.163305] loop0: detected capacity change from 0 to 1099511627776 [ 6026.294656] SGI XFS with ACLs, security attributes, quota, no debug enabled [ 6026.296737] *** VALIDATE xfs *** [ 6026.299733] XFS (loop0): Mounting V5 Filesystem [ 6026.306588] XFS (loop0): Ending clean mount [ 6026.677237] loop1: detected capacity change from 0 to 18691697672192 [ 6047.270536] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_hostid [ 6050.490359] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing load_modules_local [ 6062.526822] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6062.605518] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 6062.615628] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 6062.648886] Lustre: lustre-MDT0000: new disk, initializing [ 6062.668791] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6062.670898] Lustre: Skipped 1 previous similar message [ 6062.676362] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 6064.011772] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 6068.064641] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6068.104212] Lustre: 347627:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 6068.127073] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 6068.128724] Lustre: Skipped 1 previous similar message [ 6068.163290] Lustre: lustre-MDT0001: new disk, initializing [ 6068.204639] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 6068.207399] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 6069.503694] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 6073.099618] Lustre: DEBUG MARKER: oleg238-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 6074.749532] Lustre: DEBUG MARKER: oleg238-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 6076.934048] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6077.027537] Lustre: lustre-OST0000: new disk, initializing [ 6077.029169] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 6077.031484] Lustre: 349595:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 6078.967946] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 6078.971510] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 6079.003442] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 6079.015014] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 6082.749256] Lustre: DEBUG MARKER: oleg238-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 6084.115790] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 6084.184049] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 6085.505740] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 6085.581586] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 6088.272268] Lustre: server umount lustre-OST0000 complete [ 6093.791889] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 6093.795297] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6093.800634] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6099.424076] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6099.425883] Lustre: Skipped 3 previous similar messages [ 6100.475360] Lustre: server umount lustre-MDT0000 complete [ 6101.765166] LustreError: 347618:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788365063 with bad export cookie 12488670053779720222 [ 6101.768294] LustreError: MGC192.168.202.138@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6101.769932] LustreError: 347618:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 6112.271764] Lustre: server umount lustre-MDT0001 complete [ 6114.829150] XFS (loop0): Unmounting Filesystem [ 6116.258062] Lustre: DEBUG MARKER: == conf-sanity test 117: lctl get_param return errors properly ========================================================== 12:04:36 (1788365076) [ 6118.371599] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6118.509159] LustreError: 352149:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6118.529773] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6118.532811] Lustre: Skipped 2 previous similar messages [ 6119.762020] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 6122.460469] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6123.871156] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 6126.177986] Lustre: DEBUG MARKER: oleg238-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 6127.950543] Lustre: DEBUG MARKER: oleg238-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 6130.135618] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6132.188507] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 6134.657475] Lustre: DEBUG MARKER: oleg238-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 6149.089677] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 6149.089903] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6149.091504] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6149.092722] LustreError: Skipped 1 previous similar message [ 6149.096588] Lustre: Skipped 2 previous similar messages [ 6149.968909] Lustre: server umount lustre-OST0000 complete [ 6153.183834] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 6153.187569] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6157.267964] Lustre: server umount lustre-MDT0000 complete [ 6158.426118] LustreError: 352129:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788365119 with bad export cookie 12488670053779721104 [ 6158.428514] LustreError: MGC192.168.202.138@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6158.428946] LustreError: 352129:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 6158.546114] Lustre: server umount lustre-MDT0001 complete [ 6162.055357] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing unload_modules_local [ 6163.021876] Key type lgssc unregistered [ 6163.149332] LNet: 355797:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 6163.151816] LNetError: 355797:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 6163.160277] LNet: Removed LNI 192.168.202.138@tcp [ 6163.477099] Key type .llcrypt unregistered [ 6163.478410] Key type ._llcrypt unregistered [ 6173.787164] Key type ._llcrypt registered [ 6173.788105] Key type .llcrypt registered [ 6173.821631] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing load_modules_local [ 6174.158196] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 6174.209080] alg: No test for adler32 (adler32-zlib) [ 6175.074311] Lustre: Lustre: Build Version: 2.17.56_1_g896c54c [ 6175.172272] LNet: Added LNI 192.168.202.138@tcp [8/256/0/180] [ 6176.759177] Key type lgssc registered [ 6177.102210] Lustre: Echo OBD driver; http://www.lustre.org/ [ 6179.447181] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 6179.452726] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6180.582129] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6181.747680] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 6184.273449] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6184.383048] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 6185.612107] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 6187.810513] Lustre: DEBUG MARKER: oleg238-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 6189.486545] Lustre: DEBUG MARKER: oleg238-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 6191.626377] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6191.735219] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 6193.509362] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 6195.759484] Lustre: DEBUG MARKER: oleg238-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 6210.015818] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [ 6210.017857] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6210.022540] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 6211.501702] Lustre: lustre-MDT0001: Not available for connect from 192.168.202.38@tcp (stopping) [ 6211.555194] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6216.615804] Lustre: lustre-MDT0001: Not available for connect from 192.168.202.38@tcp (stopping) [ 6216.618981] Lustre: Skipped 2 previous similar messages [ 6221.736900] Lustre: lustre-MDT0001: Not available for connect from 192.168.202.38@tcp (stopping) [ 6221.740078] Lustre: Skipped 2 previous similar messages [ 6223.327124] Lustre: lustre-MDT0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 6223.377537] Lustre: server umount lustre-MDT0001 complete [ 6226.810037] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6226.843409] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [ 6226.845883] Lustre: Found index 1 for lustre-MDT0001, updating log [ 6226.855609] LustreError: 357334:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.202.38@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6226.858903] Lustre: 357329:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 6226.860282] LustreError: 357334:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 6226.911876] Lustre: lustre-MDT0001: Not available for connect from 0@lo (not set up) [ 6226.914347] Lustre: Skipped 3 previous similar messages [ 6226.955916] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 6228.263520] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 6232.037587] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 6234.596632] LustreError: 361121:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [ 6234.600706] LustreError: 361121:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [ 6234.605381] LustreError: 357331:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [ 6234.622991] LustreError: 361124:0:(obd_config.c:2103:class_config_llog_handler()) MGC192.168.202.138@tcp: cfg command failed: rc = -17 [ 6234.627038] Lustre: cmd=cf001 0:lustre-MDT0001-osp-MDT0000 1:osp 2:lustre-MDT0000-mdtlov_UUID [ 6234.627038] [ 6534.067361] Lustre: lustre-MDT0001: Not available for connect from 192.168.202.38@tcp (stopping) [ 6534.113028] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [ 6534.114791] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6536.873387] Lustre: server umount lustre-MDT0001 complete [ 6539.176798] LustreError: 359828:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.202.38@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6541.240836] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6541.285899] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [ 6541.288558] Lustre: Found index 1 for lustre-MDT0001, updating log [ 6541.304974] Lustre: 357327:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 6541.446251] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 6543.096424] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 6546.949221] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 6546.952267] LustreError: 373398:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [ 6546.961465] LustreError: 373398:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [ 6546.970554] LustreError: 357331:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [ 6546.974734] LustreError: 357331:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [ 6547.003270] LustreError: 373401:0:(obd_config.c:761:class_setup()) osp: device 8 already setup: rc = -17 [ 6547.006605] LustreError: 373401:0:(obd_config.c:866:class_detach()) OBD device 8 still set up [ 6547.010550] LustreError: 373401:0:(obd_config.c:2103:class_config_llog_handler()) MGC192.168.202.138@tcp: cfg command failed: rc = -17 [ 6547.015358] Lustre: cmd=cf003 0:lustre-MDT0001-osp-MDT0000 1:lustre-MDT0001_UUID 2:192.168.202.138@tcp [ 6547.015358] [ 6846.386577] Lustre: lustre-MDT0001: Not available for connect from 192.168.202.38@tcp (stopping) [ 6846.390858] Lustre: Skipped 2 previous similar messages [ 6848.996746] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [ 6848.996929] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6849.005815] Lustre: Skipped 1 previous similar message [ 6851.301111] Lustre: server umount lustre-MDT0001 complete [ 6851.496343] LustreError: 367859:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.202.38@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6851.503401] LustreError: 367859:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 6854.115473] LustreError: 358077:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6854.123803] LustreError: 358077:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 6855.494653] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6855.546596] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [ 6855.548970] Lustre: Found index 1 for lustre-MDT0001, updating log [ 6855.565566] Lustre: 357327:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 6855.705021] Lustre: lustre-MDT0001: Imperative Recovery enabled, recovery window shrunk from 60-180 down to 60-180 [ 6857.451201] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 6860.775853] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 6865.383533] LustreError: 385321:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [ 6865.387159] LustreError: 385321:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [ 6865.391785] LustreError: 357331:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [ 6865.395354] LustreError: 357331:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [ 6865.419971] LustreError: 385324:0:(lod_lov.c:210:lod_add_device()) lustre-MDT0001-osp-MDT0000: cannot connect to next dev lustre-MDT0001_UUID (-114) [ 6865.431775] LustreError: 385324:0:(obd_config.c:2103:class_config_llog_handler()) MGC192.168.202.138@tcp: cfg command failed: rc = -114 [ 6865.435458] Lustre: cmd=cf014 0:lustre-MDT0000-mdtlov 1:lustre-MDT0001_UUID 2:1 3:1 [ 6865.435458] [ 7178.211431] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7178.213691] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7178.233475] Lustre: Skipped 1 previous similar message [ 7178.238419] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7178.255225] Lustre: Skipped 4 previous similar messages [ 7179.862190] Lustre: server umount lustre-MDT0000 complete [ 7182.708161] LustreError: 357320:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788366144 with bad export cookie 8509920506124424098 [ 7182.711455] LustreError: MGC192.168.202.138@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7182.716401] LustreError: 357320:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 7183.329362] LustreError: 357335:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7192.545491] LustreError: 357335:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7192.554452] LustreError: 357335:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 7197.151252] Lustre: lustre-MDT0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 7197.406359] Lustre: server umount lustre-MDT0001 complete [ 7200.525610] Lustre: server umount lustre-OST0000 complete [ 7210.077696] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_hostid [ 7221.154525] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing load_modules_local [ 7252.312294] Lustre: DEBUG MARKER: == conf-sanity test 120: cross-target rename should not create bad symlinks ========================================================== 12:23:32 (1788366212) [ 7256.780175] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7256.903263] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 7256.920748] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 7256.970579] Lustre: lustre-MDT0000: new disk, initializing [ 7257.025114] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7257.043433] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 7260.171531] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 7266.742739] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7266.793169] Lustre: 399342:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 7266.822788] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 7266.825687] Lustre: Skipped 1 previous similar message [ 7266.865317] Lustre: lustre-MDT0001: new disk, initializing [ 7266.911338] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7266.941727] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 7266.949294] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 7269.283733] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 7274.629688] Lustre: DEBUG MARKER: oleg238-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 7277.678946] Lustre: DEBUG MARKER: oleg238-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 7281.752230] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7281.933777] Lustre: lustre-OST0000: new disk, initializing [ 7281.939378] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 7281.949437] Lustre: 401308:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 7282.020829] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7285.312680] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_default_debug -1 all [ 7287.339358] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 7287.343533] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 7287.392679] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 7290.979585] Lustre: DEBUG MARKER: oleg238-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 7297.506545] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7297.507564] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7297.511266] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7297.511274] Lustre: Skipped 4 previous similar messages [ 7297.520570] Lustre: Skipped 4 previous similar messages [ 7301.644971] Lustre: server umount lustre-MDT0000 complete [ 7302.626152] LustreError: 400153:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7302.639236] LustreError: 400153:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [ 7303.742421] LustreError: 399335:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788366265 with bad export cookie 8509920506124426037 [ 7303.743993] LustreError: MGC192.168.202.138@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7303.744256] LustreError: 400149:0:(osp_object.c:618:osp_attr_get()) lustre-MDT0000-osp-MDT0001: osp_attr_get update error [0x200000401:0x2:0x0]: rc = -5 [ 7303.746599] LustreError: 399335:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 7303.894374] Lustre: server umount lustre-MDT0001 complete [ 7316.441557] Lustre: server umount lustre-OST0000 complete [ 7331.542782] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing set_hostid [ 7336.741255] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing load_modules_local [ 7355.990234] Lustre: DEBUG MARKER: == conf-sanity test complete, duration 7038 sec ========== 12:25:15 (1788366315) [ 7357.253221] Lustre: DEBUG MARKER: === conf-sanity: start cleanup 12:25:17 (1788366317) === [ 7359.495035] Lustre: DEBUG MARKER: === conf-sanity: finish cleanup 12:25:19 (1788366319) === [ 7381.728585] Lustre: DEBUG MARKER: oleg238-server.virtnet: executing unload_modules_local [ 7383.367495] Key type lgssc unregistered [ 7383.567441] LNet: 407664:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 7383.572238] LNetError: 407664:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 7383.585130] LNet: Removed LNI 192.168.202.138@tcp [ 7384.113413] Key type .llcrypt unregistered [ 7384.116513] Key type ._llcrypt unregistered