[ 0.000000] Linux version 4.18.0rh8.10-debug (green@maintenance) (gcc version 8.5.0 20210514 (Red Hat 8.5.0-26) (GCC)) #2 SMP Mon Jul 14 01:24:22 EDT 2025 [ 0.000000] Command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers' [ 0.000000] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256 [ 0.000000] x86/fpu: Enabled xstate features 0x7, context size is 832 bytes, using 'standard' format. [ 0.000000] signal: max sigframe size: 1776 [ 0.000000] BIOS-provided physical RAM map: [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x00000000bffcdfff] usable [ 0.000000] BIOS-e820: [mem 0x00000000bffce000-0x00000000bfffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x0000000146dfffff] usable [ 0.000000] NX (Execute Disable) protection: active [ 0.000000] SMBIOS 2.8 present. [ 0.000000] DMI: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-8.fc42 06/10/2025 [ 0.000000] Hypervisor detected: KVM [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00 [ 0.000000] kvm-clock: using sched offset of 627486258 cycles [ 0.000000] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns [ 0.000000] tsc: Detected 2399.998 MHz processor [ 0.000000] last_pfn = 0x146e00 max_arch_pfn = 0x400000000 [ 0.000000] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT [ 0.000000] last_pfn = 0xbffce max_arch_pfn = 0x400000000 [ 0.000000] found SMP MP-table at [mem 0x000f54b0-0x000f54bf] [ 0.000000] RAMDISK: [mem 0xbcc54000-0xbffbffff] [ 0.000000] ACPI: Early table checksum verification disabled [ 0.000000] ACPI: RSDP 0x00000000000F52D0 000014 (v00 BOCHS ) [ 0.000000] ACPI: RSDT 0x00000000BFFE2439 000034 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACP 0x00000000BFFE22D5 000074 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: DSDT 0x00000000BFFE0040 002295 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACS 0x00000000BFFE0000 000040 [ 0.000000] ACPI: APIC 0x00000000BFFE2349 000090 (v03 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: HPET 0x00000000BFFE23D9 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: WAET 0x00000000BFFE2411 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: Reserving FACP table memory at [mem 0xbffe22d5-0xbffe2348] [ 0.000000] ACPI: Reserving DSDT table memory at [mem 0xbffe0040-0xbffe22d4] [ 0.000000] ACPI: Reserving FACS table memory at [mem 0xbffe0000-0xbffe003f] [ 0.000000] ACPI: Reserving APIC table memory at [mem 0xbffe2349-0xbffe23d8] [ 0.000000] ACPI: Reserving HPET table memory at [mem 0xbffe23d9-0xbffe2410] [ 0.000000] ACPI: Reserving WAET table memory at [mem 0xbffe2411-0xbffe2438] [ 0.000000] No NUMA configuration found [ 0.000000] Faking a node at [mem 0x0000000000000000-0x0000000146dfffff] [ 0.000000] NODE_DATA(0) allocated [mem 0x1465a3000-0x1465cdfff] [ 0.000000] Reserving 256MB of memory at 2752MB for crashkernel (System RAM: 4205MB) [ 0.000000] Zone ranges: [ 0.000000] DMA [mem 0x0000000000001000-0x0000000000ffffff] [ 0.000000] DMA32 [mem 0x0000000001000000-0x00000000ffffffff] [ 0.000000] Normal [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Device empty [ 0.000000] Movable zone start for each node [ 0.000000] Early memory node ranges [ 0.000000] node 0: [mem 0x0000000000001000-0x000000000009efff] [ 0.000000] node 0: [mem 0x0000000000100000-0x00000000bffcdfff] [ 0.000000] node 0: [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Zeroed struct page in unavailable ranges: 4756 pages [ 0.000000] Initmem setup node 0 [mem 0x0000000000001000-0x0000000146dfffff] [ 0.000000] ACPI: PM-Timer IO Port: 0x608 [ 0.000000] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1]) [ 0.000000] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23 [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level) [ 0.000000] Using ACPI (MADT) for SMP configuration information [ 0.000000] ACPI: HPET id: 0x8086a201 base: 0xfed00000 [ 0.000000] TSC deadline timer available [ 0.000000] smpboot: Allowing 4 CPUs, 0 hotplug CPUs [ 0.000000] kvm-guest: KVM setup pv remote TLB flush [ 0.000000] kvm-guest: setup PV sched yield [ 0.000000] PM: Registered nosave memory: [mem 0x00000000-0x00000fff] [ 0.000000] PM: Registered nosave memory: [mem 0x0009f000-0x0009ffff] [ 0.000000] PM: Registered nosave memory: [mem 0x000a0000-0x000effff] [ 0.000000] PM: Registered nosave memory: [mem 0x000f0000-0x000fffff] [ 0.000000] PM: Registered nosave memory: [mem 0xbffce000-0xbfffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xc0000000-0xfeffbfff] [ 0.000000] PM: Registered nosave memory: [mem 0xfeffc000-0xfeffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xff000000-0xfffbffff] [ 0.000000] PM: Registered nosave memory: [mem 0xfffc0000-0xffffffff] [ 0.000000] [mem 0xc0000000-0xfeffbfff] available for PCI devices [ 0.000000] Booting paravirtualized kernel on KVM [ 0.000000] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns [ 0.000000] setup_percpu: NR_CPUS:8192 nr_cpumask_bits:4 nr_cpu_ids:4 nr_node_ids:1 [ 0.000000] percpu: Embedded 63 pages/cpu s221184 r8192 d28672 u524288 [ 0.000000] kvm-guest: PV spinlocks enabled [ 0.000000] PV qspinlock hash table entries: 256 (order: 0, 4096 bytes, linear) [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 1059606 [ 0.000000] Policy zone: Normal [ 0.000000] Kernel command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] Specific versions of hardware are certified with Red Hat Enterprise Linux 8. Please see the list of hardware certified with Red Hat Enterprise Linux 8 at https://catalog.redhat.com. [ 0.000000] audit: disabled (until reboot) [ 0.000000] software IO TLB: area num 4. [ 0.000000] Memory: 2829652K/4306352K available (18435K kernel code, 11221K rwdata, 7248K rodata, 2908K init, 18040K bss, 524584K reserved, 0K cma-reserved) [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1 [ 0.000000] kmemleak: Kernel memory leak detector disabled [ 0.000000] ftrace: allocating 41240 entries in 162 pages [ 0.000000] ftrace: allocated 162 pages with 3 groups [ 0.000000] rcu: Hierarchical RCU implementation. [ 0.000000] rcu: RCU event tracing is enabled. [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=8192 to nr_cpu_ids=4. [ 0.000000] rcu: RCU callback double-/use-after-free debug enabled. [ 0.000000] Rude variant of Tasks RCU enabled. [ 0.000000] Tracing variant of Tasks RCU enabled. [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies. [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4 [ 0.000000] NR_IRQS: 524544, nr_irqs: 456, preallocated irqs: 16 [ 0.000000] random: get_random_bytes called from start_kernel+0x622/0x9a8 with crng_init=0 [ 0.001000] Console: colour *CGA 80x25 [ 0.001000] printk: console [ttyS1] enabled [ 0.001000] ACPI: Core revision 20220331 [ 0.001000] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns [ 0.001011] APIC: Switch to symmetric I/O mode setup [ 0.002368] x2apic enabled [ 0.003010] Switched APIC routing to physical x2apic. [ 0.004016] kvm-guest: setup PV IPIs [ 0.007000] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1 [ 0.007000] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 0.007018] Calibrating delay loop (skipped) preset value.. 4799.99 BogoMIPS (lpj=2399998) [ 0.008013] pid_max: default: 32768 minimum: 301 [ 0.009138] LSM: Security Framework initializing [ 0.010058] Yama: becoming mindful. [ 0.011040] SELinux: Initializing. [ 0.012081] *** VALIDATE selinux *** [ 0.021631] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, vmalloc) [ 0.026967] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, vmalloc) [ 0.028018] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.029315] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.030128] *** VALIDATE tmpfs *** [ 0.032248] *** VALIDATE proc *** [ 0.033431] *** VALIDATE cgroup *** [ 0.034011] *** VALIDATE cgroup2 *** [ 0.036209] x86/cpu: User Mode Instruction Prevention (UMIP) activated [ 0.038055] Last level iTLB entries: 4KB 0, 2MB 0, 4MB 0 [ 0.039012] Last level dTLB entries: 4KB 0, 2MB 0, 4MB 0, 1GB 0 [ 0.040031] Spectre V2 : User space: Vulnerable [ 0.042012] Speculative Store Bypass: Vulnerable [ 0.046211] debug: unmapping init [mem 0xffffffff91e59000-0xffffffff91e60fff] [ 0.049000] smpboot: CPU0: Intel(R) Xeon(R) CPU E5-2695 v2 @ 2.40GHz (family: 0x6, model: 0x3e, stepping: 0x4) [ 0.049958] Performance Events: IvyBridge events, full-width counters, Intel PMU driver. [ 0.051028] ... version: 2 [ 0.052015] ... bit width: 48 [ 0.053014] ... generic registers: 4 [ 0.054013] ... value mask: 0000ffffffffffff [ 0.055016] ... max period: 00007fffffffffff [ 0.056015] ... fixed-purpose events: 3 [ 0.057013] ... event mask: 000000070000000f [ 0.058288] rcu: Hierarchical SRCU implementation. [ 0.061382] smp: Bringing up secondary CPUs ... [ 0.063038] x86: Booting SMP configuration: [ 0.064016] .... node #0, CPUs: #1 #2 #3 [ 0.068655] smp: Brought up 1 node, 4 CPUs [ 0.070032] smpboot: Max logical packages: 1 [ 0.071013] smpboot: Total of 4 processors activated (19199.98 BogoMIPS) [ 0.179025] node 0 deferred pages initialised in 105ms [ 0.183388] devtmpfs: initialized [ 0.184204] x86/mm: Memory block size: 128MB [ 0.187421] gcov: version magic: 0x41383552 [ 0.189345] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns [ 0.193069] futex hash table entries: 1024 (order: 4, 65536 bytes, vmalloc) [ 0.195444] pinctrl core: initialized pinctrl subsystem [ 0.197108] [ 0.197711] ************************************************************* [ 0.200014] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.203060] ** ** [ 0.205011] ** IOMMU DebugFS SUPPORT HAS BEEN ENABLED IN THIS KERNEL ** [ 0.207015] ** ** [ 0.211013] ** This means that this kernel is built to expose internal ** [ 0.214013] ** IOMMU data structures, which may compromise security on ** [ 0.217011] ** your system. ** [ 0.219016] ** ** [ 0.222014] ** If you see this message and you are not debugging the ** [ 0.225013] ** kernel, report this immediately to your vendor! ** [ 0.228012] ** ** [ 0.230015] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.233011] ************************************************************* [ 0.237882] NET: Registered protocol family 16 [ 0.239586] DMA: preallocated 512 KiB GFP_KERNEL pool for atomic allocations [ 0.242047] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations [ 0.244048] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations [ 0.248011] cpuidle: using governor menu [ 0.249921] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5 [ 0.250454] PCI: Using configuration type 1 for base access [ 0.253125] core: PMU erratum BJ122, BV98, HSD29 worked around, HT is on [ 0.261053] HugeTLB registered 1.00 GiB page size, pre-allocated 0 pages [ 0.264020] HugeTLB registered 2.00 MiB page size, pre-allocated 0 pages [ 0.267084] cryptd: max_cpu_qlen set to 1000 [ 0.270286] ACPI: Added _OSI(Module Device) [ 0.272010] ACPI: Added _OSI(Processor Device) [ 0.273009] ACPI: Added _OSI(3.0 _SCP Extensions) [ 0.276017] ACPI: Added _OSI(Processor Aggregator Device) [ 0.281486] ACPI: 1 ACPI AML tables successfully acquired and loaded [ 0.288542] ACPI: Interpreter enabled [ 0.289000] ACPI: PM: (supports S0 S3 S4 S5) [ 0.290015] ACPI: Using IOAPIC for interrupt routing [ 0.293120] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug [ 0.299814] ACPI: Enabled 2 GPEs in block 00 to 0F [ 0.311769] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff]) [ 0.315045] acpi PNP0A03:00: _OSC: OS supports [ASPM ClockPM Segments MSI HPX-Type3] [ 0.319020] acpi PNP0A03:00: _OSC: not requesting OS control; OS requires [ExtendedConfig ASPM ClockPM MSI] [ 0.322066] acpi PNP0A03:00: fail to add MMCONFIG information, can't access extended PCI configuration space under this bridge. [ 0.327551] acpiphp: Slot [2] registered [ 0.329473] acpiphp: Slot [5] registered [ 0.332184] acpiphp: Slot [6] registered [ 0.333153] acpiphp: Slot [7] registered [ 0.334209] acpiphp: Slot [8] registered [ 0.336118] acpiphp: Slot [9] registered [ 0.337098] acpiphp: Slot [10] registered [ 0.339143] acpiphp: Slot [3] registered [ 0.340112] acpiphp: Slot [4] registered [ 0.342100] acpiphp: Slot [11] registered [ 0.344123] acpiphp: Slot [12] registered [ 0.346306] acpiphp: Slot [13] registered [ 0.348383] acpiphp: Slot [14] registered [ 0.350083] acpiphp: Slot [15] registered [ 0.352090] acpiphp: Slot [16] registered [ 0.353388] acpiphp: Slot [17] registered [ 0.355314] acpiphp: Slot [18] registered [ 0.357101] acpiphp: Slot [19] registered [ 0.359151] acpiphp: Slot [20] registered [ 0.361086] acpiphp: Slot [21] registered [ 0.363137] acpiphp: Slot [22] registered [ 0.365130] acpiphp: Slot [23] registered [ 0.367112] acpiphp: Slot [24] registered [ 0.369143] acpiphp: Slot [25] registered [ 0.370107] acpiphp: Slot [26] registered [ 0.371086] acpiphp: Slot [27] registered [ 0.373290] acpiphp: Slot [28] registered [ 0.375074] acpiphp: Slot [29] registered [ 0.376076] acpiphp: Slot [30] registered [ 0.377090] acpiphp: Slot [31] registered [ 0.378057] PCI host bridge to bus 0000:00 [ 0.380023] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window] [ 0.382022] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window] [ 0.384021] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window] [ 0.387024] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window] [ 0.390025] pci_bus 0000:00: root bus resource [mem 0xe0000000000-0xe007fffffff window] [ 0.394033] pci_bus 0000:00: root bus resource [bus 00-ff] [ 0.397212] pci 0000:00:00.0: [8086:1237] type 00 class 0x060000 [ 0.400218] pci 0000:00:01.0: [8086:7000] type 00 class 0x060100 [ 0.404300] pci 0000:00:01.1: [8086:7010] type 00 class 0x010180 [ 0.417017] pci 0000:00:01.1: reg 0x20: [io 0xc320-0xc32f] [ 0.423056] pci 0000:00:01.1: legacy IDE quirk: reg 0x10: [io 0x01f0-0x01f7] [ 0.427026] pci 0000:00:01.1: legacy IDE quirk: reg 0x14: [io 0x03f6] [ 0.430019] pci 0000:00:01.1: legacy IDE quirk: reg 0x18: [io 0x0170-0x0177] [ 0.434031] pci 0000:00:01.1: legacy IDE quirk: reg 0x1c: [io 0x0376] [ 0.437677] pci 0000:00:01.3: [8086:7113] type 00 class 0x068000 [ 0.442102] pci 0000:00:01.3: quirk: [io 0x0600-0x063f] claimed by PIIX4 ACPI [ 0.446052] pci 0000:00:01.3: quirk: [io 0x0700-0x070f] claimed by PIIX4 SMB [ 0.448647] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 [ 0.453015] pci 0000:00:02.0: reg 0x10: [io 0xc300-0xc31f] [ 0.469016] pci 0000:00:02.0: reg 0x20: [mem 0xe0000000000-0xe0000003fff 64bit pref] [ 0.477014] pci 0000:00:02.0: reg 0x30: [mem 0xfeb80000-0xfebbffff pref] [ 0.483691] pci 0000:00:05.0: [1af4:1001] type 00 class 0x010000 [ 0.492018] pci 0000:00:05.0: reg 0x10: [io 0xc000-0xc07f] [ 0.499018] pci 0000:00:05.0: reg 0x14: [mem 0xfebc0000-0xfebc0fff] [ 0.522019] pci 0000:00:05.0: reg 0x20: [mem 0xe0000004000-0xe0000007fff 64bit pref] [ 0.538654] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 [ 0.545018] pci 0000:00:06.0: reg 0x10: [io 0xc080-0xc0ff] [ 0.552016] pci 0000:00:06.0: reg 0x14: [mem 0xfebc1000-0xfebc1fff] [ 0.568019] pci 0000:00:06.0: reg 0x20: [mem 0xe0000008000-0xe000000bfff 64bit pref] [ 0.577371] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 [ 0.585016] pci 0000:00:07.0: reg 0x10: [io 0xc100-0xc17f] [ 0.592019] pci 0000:00:07.0: reg 0x14: [mem 0xfebc2000-0xfebc2fff] [ 0.609019] pci 0000:00:07.0: reg 0x20: [mem 0xe000000c000-0xe000000ffff 64bit pref] [ 0.620853] pci 0000:00:08.0: [1af4:1001] type 00 class 0x010000 [ 0.628016] pci 0000:00:08.0: reg 0x10: [io 0xc180-0xc1ff] [ 0.636018] pci 0000:00:08.0: reg 0x14: [mem 0xfebc3000-0xfebc3fff] [ 0.651034] pci 0000:00:08.0: reg 0x20: [mem 0xe0000010000-0xe0000013fff 64bit pref] [ 0.663690] pci 0000:00:09.0: [1af4:1001] type 00 class 0x010000 [ 0.672016] pci 0000:00:09.0: reg 0x10: [io 0xc200-0xc27f] [ 0.678016] pci 0000:00:09.0: reg 0x14: [mem 0xfebc4000-0xfebc4fff] [ 0.696017] pci 0000:00:09.0: reg 0x20: [mem 0xe0000014000-0xe0000017fff 64bit pref] [ 0.707514] pci 0000:00:0a.0: [1af4:1001] type 00 class 0x010000 [ 0.714027] pci 0000:00:0a.0: reg 0x10: [io 0xc280-0xc2ff] [ 0.723024] pci 0000:00:0a.0: reg 0x14: [mem 0xfebc5000-0xfebc5fff] [ 0.741028] pci 0000:00:0a.0: reg 0x20: [mem 0xe0000018000-0xe000001bfff 64bit pref] [ 0.758390] ACPI: PCI: Interrupt link LNKA configured for IRQ 10 [ 0.760226] ACPI: PCI: Interrupt link LNKB configured for IRQ 10 [ 0.762302] ACPI: PCI: Interrupt link LNKC configured for IRQ 11 [ 0.765287] ACPI: PCI: Interrupt link LNKD configured for IRQ 11 [ 0.767145] ACPI: PCI: Interrupt link LNKS configured for IRQ 9 [ 0.772312] iommu: Default domain type: Passthrough [ 0.773734] SCSI subsystem initialized [ 0.775160] ACPI: bus type USB registered [ 0.776335] usbcore: registered new interface driver usbfs [ 0.779085] usbcore: registered new interface driver hub [ 0.780077] usbcore: registered new device driver usb [ 0.782149] pps_core: LinuxPPS API ver. 1 registered [ 0.784010] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti [ 0.788086] PTP clock support registered [ 0.791130] EDAC MC: Ver: 3.0.0 [ 0.793509] PCI: Using ACPI for IRQ routing [ 0.795035] NetLabel: Initializing [ 0.797223] NetLabel: domain hash size = 128 [ 0.799016] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO [ 0.800104] NetLabel: unlabeled traffic allowed by default [ 0.803073] vgaarb: loaded [ 0.804332] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0 [ 0.806015] hpet0: 3 comparators, 64-bit 100.000000 MHz counter [ 0.811430] clocksource: Switched to clocksource kvm-clock [ 0.934221] VFS: Disk quotas dquot_6.6.0 [ 0.936163] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) [ 0.939355] *** VALIDATE ramfs *** [ 0.940861] *** VALIDATE hugetlbfs *** [ 0.942920] pnp: PnP ACPI init [ 0.945610] pnp: PnP ACPI: found 6 devices [ 0.966203] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns [ 0.970306] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window] [ 0.973293] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window] [ 0.976020] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window] [ 0.979269] pci_bus 0000:00: resource 7 [mem 0xc0000000-0xfebfffff window] [ 0.983057] pci_bus 0000:00: resource 8 [mem 0xe0000000000-0xe007fffffff window] [ 0.986356] NET: Registered protocol family 2 [ 0.989594] IP idents hash table entries: 131072 (order: 8, 1048576 bytes, vmalloc) [ 0.996539] tcp_listen_portaddr_hash hash table entries: 4096 (order: 5, 163840 bytes, vmalloc) [ 1.002382] TCP established hash table entries: 65536 (order: 7, 524288 bytes, vmalloc) [ 1.008403] TCP bind hash table entries: 65536 (order: 9, 2097152 bytes, vmalloc) [ 1.014165] TCP: Hash tables configured (established 65536 bind 65536) [ 1.018897] MPTCP token hash table entries: 8192 (order: 6, 393216 bytes, vmalloc) [ 1.022257] UDP hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 1.025081] UDP-Lite hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 1.027964] NET: Registered protocol family 1 [ 1.031694] RPC: Registered named UNIX socket transport module. [ 1.034233] RPC: Registered udp transport module. [ 1.035905] RPC: Registered tcp transport module. [ 1.037633] RPC: Registered tcp NFSv4.1 backchannel transport module. [ 1.040407] NET: Registered protocol family 44 [ 1.041925] pci 0000:00:00.0: Limiting direct PCI/PCI transfers [ 1.044606] pci 0000:00:01.0: PIIX3: Enabling Passive Release [ 1.047734] pci 0000:00:01.0: Activating ISA DMA hang workarounds [ 1.051024] PCI: CLS 0 bytes, default 64 [ 1.054594] Unpacking initramfs... [ 2.772142] debug: unmapping init [mem 0xffff9d8ebcc54000-0xffff9d8ebffbffff] [ 2.775074] PCI-DMA: Using software bounce buffering for IO (SWIOTLB) [ 2.778138] software IO TLB: mapped [mem 0x00000000a8000000-0x00000000ac000000] (64MB) [ 2.782034] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 3.434196] Initialise system trusted keyrings [ 3.436843] Key type blacklist registered [ 3.438957] workingset: timestamp_bits=36 max_order=20 bucket_order=0 [ 3.450703] zbud: loaded [ 3.454482] *** VALIDATE nfs *** [ 3.456107] *** VALIDATE nfs4 *** [ 3.457876] pstore: using deflate compression [ 3.462125] Platform Keyring initialized [ 3.580144] NET: Registered protocol family 38 [ 3.582273] Key type asymmetric registered [ 3.584383] Asymmetric key parser 'x509' registered [ 3.586267] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 247) [ 3.589588] io scheduler mq-deadline registered [ 3.592077] io scheduler kyber registered [ 3.594433] io scheduler bfq registered [ 3.597114] atomic64_test: passed for x86-64 platform with CX8 and with SSE [ 3.601231] shpchp: Standard Hot Plug PCI Controller Driver version: 0.4 [ 3.605355] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0 [ 3.608416] ACPI: Power Button [PWRF] [ 3.615831] ACPI: \_SB_.LNKB: Enabled at IRQ 10 [ 3.624730] ACPI: \_SB_.LNKA: Enabled at IRQ 11 [ 3.642724] ACPI: \_SB_.LNKC: Enabled at IRQ 11 [ 3.660206] ACPI: \_SB_.LNKD: Enabled at IRQ 10 [ 3.680588] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled [ 3.709847] 00:03: ttyS1 at I/O 0x2f8 (irq = 3, base_baud = 115200) is a 16550A [ 3.745916] 00:04: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A [ 3.751516] Non-volatile memory driver v1.3 [ 3.753927] Linux agpgart interface v0.103 [ 3.793995] virtio_blk virtio1: [vda] 135128 512-byte logical blocks (69.2 MB/66.0 MiB) [ 3.797501] vda: detected capacity change from 0 to 69185536 [ 3.817891] virtio_blk virtio2: [vdb] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) [ 3.821778] vdb: detected capacity change from 0 to 1073741824 [ 3.843675] virtio_blk virtio3: [vdc] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 3.847562] vdc: detected capacity change from 0 to 2621440000 [ 3.866724] virtio_blk virtio4: [vdd] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 3.871395] vdd: detected capacity change from 0 to 2621440000 [ 3.888055] virtio_blk virtio5: [vde] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 3.891858] vde: detected capacity change from 0 to 4294967296 [ 3.913985] virtio_blk virtio6: [vdf] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 3.917697] vdf: detected capacity change from 0 to 4294967296 [ 3.924540] libphy: Fixed MDIO Bus: probed [ 3.932669] usbcore: registered new interface driver usbserial_generic [ 3.936752] usbserial: USB Serial support registered for generic [ 3.939773] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12 [ 3.945766] serio: i8042 KBD port at 0x60,0x64 irq 1 [ 3.948190] serio: i8042 AUX port at 0x60,0x64 irq 12 [ 3.950335] mousedev: PS/2 mouse device common for all mice [ 3.954399] rtc_cmos 00:05: RTC can wake from S4 [ 3.956253] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input1 [ 3.961327] rtc_cmos 00:05: registered as rtc0 [ 3.961575] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input4 [ 3.963468] rtc_cmos 00:05: alarms up to one day, y3k, 242 bytes nvram, hpet irqs [ 3.968245] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input3 [ 3.969606] intel_pstate: CPU model not supported [ 3.976811] hid: raw HID events driver (C) Jiri Kosina [ 3.979228] usbcore: registered new interface driver usbhid [ 3.982263] usbhid: USB HID core driver [ 3.983810] drop_monitor: Initializing network drop monitor service [ 3.986391] Initializing XFRM netlink socket [ 3.988377] NET: Registered protocol family 10 [ 3.991064] Segment Routing with IPv6 [ 3.992969] NET: Registered protocol family 17 [ 3.996137] mpls_gso: MPLS GSO support [ 4.001954] RAS: Correctable Errors collector initialized. [ 4.004183] AVX version of gcm_enc/dec engaged. [ 4.005565] AES CTR mode by8 optimization enabled [ 4.089106] sched_clock: Marking stable (4089057426, 0)->(5252891955, -1163834529) [ 4.095239] registered taskstats version 1 [ 4.097553] Loading compiled-in X.509 certificates [ 4.099729] zswap: loaded using pool lzo/zbud [ 4.133580] Key type big_key registered [ 4.149551] Key type encrypted registered [ 4.152741] ima: No TPM chip found, activating TPM-bypass! [ 4.154921] ima: Allocated hash algorithm: sha1 [ 4.156948] ima: No architecture policies found [ 4.158831] evm: Initialising EVM extended attributes: [ 4.160744] evm: security.selinux [ 4.161908] evm: security.ima [ 4.163526] evm: security.capability [ 4.165184] evm: HMAC attrs: 0x1 [ 4.167878] rtc_cmos 00:05: setting system clock to 2026-03-10 00:23:08 UTC (1773102188) [ 4.174522] debug: unmapping init [mem 0xffffffff92e03000-0xffffffff92ffffff] [ 4.177922] debug: unmapping init [mem 0xffffffff91b82000-0xffffffff91e58fff] [ 4.186181] Write protecting the kernel read-only data: 28672k [ 4.190374] debug: unmapping init [mem 0xffffffff90203000-0xffffffff903fffff] [ 4.193110] debug: unmapping init [mem 0xffffffff90b14000-0xffffffff90bfffff] [ 4.240544] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 4.250797] systemd[1]: Detected virtualization kvm. [ 4.253103] systemd[1]: Detected architecture x86-64. [ 4.255771] systemd[1]: Running in initial RAM disk. Welcome to Rocky Linux 8.10 (Green Obsidian) dracut-049-233.git20240115.el8 (Initramfs)! [ 4.283453] systemd[1]: No hostname configured. [ 4.285532] systemd[1]: Set hostname to . [ 4.288164] random: systemd: uninitialized urandom read (16 bytes read) [ 4.291939] systemd[1]: Initializing machine ID from random generator. [ 4.467650] random: systemd: uninitialized urandom read (16 bytes read) [ 4.470801] systemd[1]: Listening on Journal Socket (/dev/log). [ OK ] Listening on Journal Socket (/dev/log). [ 4.478286] random: systemd: uninitialized urandom read (16 bytes read) [ 4.483089] systemd[1]: Reached target Local File Systems. [ OK ] Reached target Local File Systems. [ 4.490260] systemd[1]: Listening on udev Control Socket. [ OK ] Listening on udev Control Socket. [ OK ] Reached target Swap. [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Listening on Journal Socket. Starting Create Volatile Files and Directories... [ OK ] Reached target Paths. [ OK ] Reached target Local Encrypted Volumes. Starting Create list of required st…ce nodes for the current kernel... [ OK ] Reached target Slices. Starting Journal Service... [ OK ] Listening on udev Kernel Socket. [ OK ] Reached target Sockets. [ OK ] Reached target Initrd Root Device. [ OK ] Reached target Timers. Starting Setup Virtual Console... Starting Apply Kernel Variables... [ OK ] Started Memstrack Anylazing Service. [ OK ] Started Create Volatile Files and Directories. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Started Setup Virtual Console. [ OK ] Started Apply Kernel Variables. Starting dracut cmdline hook... Starting Create Static Device Nodes in /dev... [ OK ] Started Create Static Device Nodes in /dev. [ OK ] Started Journal Service. [ OK ] Started dracut cmdline hook. Starting dracut pre-udev hook... [ 5.328073] device-mapper: uevent: version 1.0.3 [ 5.330698] device-mapper: ioctl: 4.46.0-ioctl (2022-02-22) initialised: dm-devel@redhat.com [ OK ] Started dracut pre-udev hook. Starting udev Kernel Device Manager... [ OK ] Started udev Kernel Device Manager. Starting dracut pre-trigger hook... [ OK ] Started dracut pre-trigger hook. Starting udev Coldplug all Devices... Mounting Kernel Configuration File System... [ OK ] Mounted Kernel Configuration File System. [ OK ] Started udev Coldplug all Devices. Starting dracut initqueue hook... [ OK ] Reached target System Initialization. [ OK ] Reached target Basic System. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. [ 6.257818] virtio_net virtio0 ens2: renamed from eth0 [ 6.288816] random: fast init done [ 6.321639] scsi host0: ata_piix [ 6.337175] scsi host1: ata_piix [ 6.339367] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc320 irq 14 [ 6.342173] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc328 irq 15 [ 10.671450] random: crng init done [ 10.673365] random: 7 urandom warning(s) missed due to ratelimiting [ 11.081658] dracut-initqueue[582]: RTNETLINK answers: File exists Starting nbd nbd0... [ OK ] Started nbd nbd0. [ OK ] Started dracut initqueue hook. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Mounting /sysroot... [ 11.734669] EXT4-fs (nbd0): mounted filesystem with ordered data mode. Opts: (null) [ OK ] Mounted /sysroot. [ OK ] Reached target Initrd Root File System. Starting Reload Configuration from the Real Root... [ OK ] Started Reload Configuration from the Real Root. [ OK ] Reached target Initrd File Systems. [ OK ] Reached target Initrd Default Target. Starting dracut pre-pivot and cleanup hook... [ OK ] Started dracut pre-pivot and cleanup hook. Starting Cleaning Up and Shutting Down Daemons... Stopping Hardware RNG Entropy Gatherer Daemon... [ OK ] Stopped dracut pre-pivot and cleanup hook. [ OK ] Stopped target Remote File Systems. [ OK ] Stopped target Initrd Default Target. [ OK ] Stopped target Initrd Root Device. [ OK ] Stopped target Remote File Systems (Pre). [ OK ] Stopped dracut initqueue hook. [ OK ] Stopped target Timers. [ OK ] Stopped Hardware RNG Entropy Gatherer Daemon. [ OK ] Stopped target Basic System. [ OK ] Stopped target Slices. [ OK ] Stopped target Paths. [ OK ] Stopped target Sockets. [ OK ] Stopped target System Initialization. [ OK ] Stopped target Swap. [ OK ] Stopped target Local Encrypted Volumes. [ OK ] Stopped Dispatch Password Requests to Console Directory Watch. [ OK ] Stopped udev Coldplug all Devices. [ OK ] Stopped dracut pre-trigger hook. Stopping udev Kernel Device Manager... [ OK ] Stopped Create Volatile Files and Directories. [ OK ] Stopped target Local File Systems. [ OK ] Stopped Apply Kernel Variables. [ OK ] Stopped udev Kernel Device Manager. [ OK ] Started Cleaning Up and Shutting Down Daemons. [ OK ] Stopped dracut pre-udev hook. [ OK ] Stopped dracut cmdline hook. [ OK ] Stopped Create Static Device Nodes in /dev. [ OK ] Stopped Create list of required sta…vice nodes for the current kernel. [ OK ] Closed udev Control Socket. [ OK ] Closed udev Kernel Socket. Starting Cleanup udevd DB... [ OK ] Started Cleanup udevd DB. [ OK ] Reached target Switch Root. Starting Switch Root... [ 13.218062] printk: systemd: 25 output lines suppressed due to ratelimiting [ 13.530813] SELinux: Disabled at runtime. [ 13.603146] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 13.610286] systemd[1]: Detected virtualization kvm. [ 13.612066] systemd[1]: Detected architecture x86-64. Welcome to Rocky Linux 8.10 (Green Obsidian)! [ 14.196689] systemd[1]: initrd-switch-root.service: Succeeded. [ 14.200239] systemd[1]: Stopped Switch Root. [ OK ] Stopped Switch Root. [ 14.205601] systemd[1]: systemd-journald.service: Service has no hold-off time (RestartSec=0), scheduling restart. [ 14.211671] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. [ 14.217709] systemd[1]: Stopped Journal Service. [ OK ] Stopped Journal Service. [ 14.229740] systemd[1]: Starting Journal Service... Starting Journal Service... [ 14.236664] systemd[1]: Created slice User and Session Slice. [ OK ] Created slice User and Session Slice. [ OK ] Listening on udev Control Socket. Starting Remount Root and Kernel File Systems... [ OK ] Created slice system-serial\x2dgetty.slice. [ OK ] Listening on RPCbind Server Activation Socket. [ OK ] Started Forward Password Requests to Wall Directory Watch. Mounting Huge Pages File System... Mounting POSIX Message Queue File System... Starting Create list of required st…ce nodes for the current kernel... [ OK ] Listening on initctl Compatibility Named Pipe. [ OK ] Reached target Slices. Mounting Kernel Debug File System... Activating swap /dev/disk/by-label/SWAP... [ OK ] Reached target RPC Port Mapper. [ OK ] Started Dispatch Password Requests to Console Directory Watch. [FAILED] Failed to set up automount Arbitrar…rmats File System Automount Point. See 'systemctl status proc-sys-fs-binfmt_misc.automount' for details. [ OK ] Listening on udev Kernel Socket. Starting udev Coldplug all Devices... Starting Apply Kernel Variables... [ 14.375618] Adding 1048572k swap on /dev/vdb. Priority:-2 extents:1 across:1048572k FS [ OK ] Created slice system-sshd\x2dkeygen.slice. [ OK ] Stopped target Switch Root. [ OK ] Stopped target Initrd File Systems. [ OK ] Stopped target Initrd Root File System. [ OK ] Reached target Paths. [ OK ] Listening on Process Core Dump Socket. [ OK ] Created slice system-getty.slice. [ OK ] Reached target rpc_pipefs.target. [ OK ] Reached target Local Encrypted Volumes. [ OK ] Started Journal Service. [FAILED] Failed to start Remount Root and Kernel File Systems. See 'systemctl status systemd-remount-fs.service' for details. [ OK ] Mounted Huge Pages File System. [ OK ] Mounted POSIX Message Queue File System. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Mounted Kernel Debug File System. [ OK ] Activated swap /dev/disk/by-label/SWAP. [ OK ] Started Apply Kernel Variables. [ OK ] Reached target Swap. Starting Configure read-only root support... Starting Create Static Device Nodes in /dev... Starting Flush Journal to Persistent Storage... [ OK ] Started Flush Journal to Persistent Storage. [ OK ] Started Create Static Device Nodes in /dev. Starting udev Kernel Device Manager... [ OK ] Reached target Local File Systems (Pre). Mounting /mnt... Mounting /home/green/git/lustre-release... [ OK ] Mounted /mnt. [ OK ] Started udev Coldplug all Devices. [ 14.763160] squashfs: version 4.0 (2009/01/31) Phillip Lougher [ OK ] Mounted /home/green/git/lustre-release. [ OK ] Started udev Kernel Device Manager. [ 15.101135] piix4_smbus 0000:00:01.3: SMBus Host Controller at 0x700, revision 0 [ 15.165680] input: PC Speaker as /devices/platform/pcspkr/input/input5 [ 15.268315] RAPL PMU: API unit is 2^-32 Joules, 0 fixed counters, 10737418240 ms ovfl timer [ 15.284290] EDAC sbridge: Ver: 1.1.2 [ 17.307736] Key type dns_resolver registered [ 17.712170] NFS: Registering the id_resolver key type [ 17.714892] Key type id_resolver registered [ 17.716885] Key type id_legacy registered [ OK ] Started Configure read-only root support. [ OK ] Reached target Local File Systems. Starting Rebuild Dynamic Linker Cache... Starting Create Volatile Files and Directories... Starting Mark the need to relabel after reboot... Starting Load/Save Random Seed... [ OK ] Started Mark the need to relabel after reboot. [ OK ] Started Load/Save Random Seed. [ OK ] Started Create Volatile Files and Directories. Starting RPC Bind... Starting Update UTMP about System Boot/Shutdown... [ OK ] Started Update UTMP about System Boot/Shutdown. [ OK ] Started RPC Bind. [ OK ] Started Rebuild Dynamic Linker Cache. Starting Update is Completed... [ OK ] Started Update is Completed. [ OK ] Reached target System Initialization. [ OK ] Listening on D-Bus System Message Bus Socket. [ OK ] Reached target Sockets. [ OK ] Reached target Basic System. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. [ OK ] Started dnf makecache --timer. [ OK ] Started irqbalance daemon. Starting Login Service... Starting Restore /run/initramfs on shutdown... [ OK ] Started D-Bus System Message Bus. Starting Network Manager... [ OK ] Started daily update of the root trust anchor for DNSSEC. [ OK ] Reached target sshd-keygen.target. [ OK ] Started Daily Cleanup of Temporary Directories. [ OK ] Reached target Timers. [ OK ] Started Restore /run/initramfs on shutdown. [ OK ] Started Login Service. [ OK ] Started Network Manager. Starting Network Manager Wait Online... [ OK ] Reached target Network. Starting OpenSSH server daemon... Starting Dynamic System Tuning Daemon... Starting GSSAPI Proxy Daemon... Starting Hostname Service... [ OK ] Started OpenSSH server daemon. [ OK ] Started GSSAPI Proxy Daemon. [ OK ] Reached target NFS client services. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Starting Permit User Sessions... [ OK ] Started Permit User Sessions. [ OK ] Started Command Scheduler. [ OK ] Started Serial Getty on ttyS1. [ OK ] Started Serial Getty on ttyS0. [ OK ] Started Getty on tty1. [ OK ] Reached target Login Prompts. [ OK ] Started Hostname Service. Starting Network Manager Script Dispatcher Service... [ OK ] Started Network Manager Script Dispatcher Service. [ OK ] Started Network Manager Wait Online. [ OK ] Reached target Network is Online. Starting Notify NFS peers of a restart... Starting Crash recovery kernel arming... Starting System Logging Service... [ OK ] Started Notify NFS peers of a restart. [ OK ] Started System Logging Service. Starting Authorization Manager... [ OK ] Started Authorization Manager. [ OK ] Started Dynamic System Tuning Daemon. [ OK ] Reached target Multi-User System. [ OK ] Reached target Graphical Interface. Starting Update UTMP about System Runlevel Changes... [ OK ] Started Update UTMP about System Runlevel Changes. Rocky Linux 8.10 (Green Obsidian) Kernel 4.18.0rh8.10-debug on an x86_64 oleg147-server login: [ 62.742252] libcfs: loading out-of-tree module taints kernel. [ 62.790661] Key type ._llcrypt registered [ 62.792289] Key type .llcrypt registered [ 62.902987] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing set_hostid [ 77.479900] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing load_modules_local [ 78.825061] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 78.850531] alg: No test for adler32 (adler32-zlib) [ 80.334908] Lustre: Lustre: Build Version: 2.17.50_236_gea5aa0c [ 81.041667] LNet: Added LNI 192.168.201.147@tcp [8/256/0/180] [ 82.767264] Key type lgssc registered [ 83.977177] Lustre: Echo OBD driver; http://www.lustre.org/ [ 96.984496] ZFS: Loaded module v2.3.2-1, ZFS pool version 5000, ZFS filesystem version 5 [ 130.034806] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing load_modules_local [ 139.758828] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 139.787188] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 141.031113] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 141.064940] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 141.136388] Lustre: lustre-MDT0000: new disk, initializing [ 141.222282] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 141.244671] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 144.599447] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 155.726761] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 155.803684] Lustre: 6493:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 155.837919] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 155.840770] Lustre: Skipped 1 previous similar message [ 155.930678] Lustre: lustre-MDT0001: new disk, initializing [ 155.991569] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 156.020270] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 156.033940] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 159.164773] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 163.410530] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 170.863415] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 171.073442] Lustre: lustre-OST0000: new disk, initializing [ 171.077772] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 171.130985] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 175.671697] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 179.772079] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 179.787265] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 179.873226] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 188.968050] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 189.061186] Lustre: lustre-OST0001: new disk, initializing [ 189.066207] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 189.117288] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 194.196896] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 195.602500] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 195.615886] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 195.716324] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 204.384647] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 209.287428] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 216.798079] hrtimer: interrupt took 5688181 ns [ 219.287937] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing check_logdir /tmp/testlogs/ [ 223.451268] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing yml_node [ 226.971626] Lustre: DEBUG MARKER: Client: 2.17.50.236 [ 229.318936] Lustre: DEBUG MARKER: MDS: 2.17.50.236 [ 231.510350] Lustre: DEBUG MARKER: OSS: 2.17.50.236 [ 232.805755] Lustre: DEBUG MARKER: -----============= acceptance-small: conf-sanity ============----- Mon Mar 9 20:26:56 EDT 2026 [ 247.637858] Lustre: DEBUG MARKER: excepting tests: 102 106 115 32newtarball [ 249.028121] Lustre: DEBUG MARKER: skipping tests SLOW=no: 45 69 106 111 114 [ 263.648748] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 263.656381] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 263.671532] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 267.237067] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 267.241756] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 267.245381] Lustre: Skipped 2 previous similar messages [ 267.273922] Lustre: Skipped 2 previous similar messages [ 269.515508] Lustre: server umount lustre-MDT0000 complete [ 272.407665] LustreError: 7872:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1773102456 with bad export cookie 15142586707670905996 [ 272.410184] LustreError: MGC192.168.201.147@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 272.420571] LustreError: 7872:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 277.475134] LustreError: 6500:0:(ldlm_lib.c:1178:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 277.476173] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 277.478726] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 277.478735] Lustre: Skipped 1 previous similar message [ 277.499628] LustreError: 6500:0:(ldlm_lib.c:1178:target_handle_connect()) Skipped 6 previous similar messages [ 277.506501] Lustre: Skipped 1 previous similar message [ 278.787747] Lustre: server umount lustre-MDT0001 complete [ 282.219431] Lustre: server umount lustre-OST0000 complete [ 286.072460] Lustre: server umount lustre-OST0001 complete [ 291.845399] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing set_hostid [ 299.302801] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing load_modules_local [ 327.945616] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 328.156693] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 328.178496] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 328.226844] Lustre: lustre-MDT0000: new disk, initializing [ 328.295711] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 328.318180] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 331.004249] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing set_default_debug -1 all [ 339.148586] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 339.233901] Lustre: 14956:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 339.276899] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 339.281462] Lustre: Skipped 1 previous similar message [ 339.370367] Lustre: lustre-MDT0001: new disk, initializing [ 339.468273] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 339.514112] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 339.527357] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 343.412581] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing set_default_debug -1 all [ 349.495513] Lustre: DEBUG MARKER: oleg147-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 352.237219] Lustre: DEBUG MARKER: oleg147-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 356.518880] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 356.790642] Lustre: lustre-OST0000: new disk, initializing [ 356.793226] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 356.842381] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 358.622833] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 358.633196] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 358.701517] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 361.002385] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing set_default_debug -1 all [ 366.400355] Lustre: DEBUG MARKER: oleg147-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 369.001456] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 369.152755] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 371.881668] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 372.014254] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 374.244222] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 374.257039] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 374.270407] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 374.275918] Lustre: Skipped 2 previous similar messages [ 379.397910] Lustre: server umount lustre-OST0000 complete [ 382.433104] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 382.435022] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 382.440292] Lustre: Skipped 2 previous similar messages [ 382.447474] Lustre: Skipped 2 previous similar messages [ 388.621398] Lustre: server umount lustre-MDT0000 complete [ 391.168886] LustreError: 16700:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1773102575 with bad export cookie 15142586707670907529 [ 391.170695] LustreError: MGC192.168.201.147@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 391.178881] LustreError: 16700:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 391.451508] Lustre: server umount lustre-MDT0001 complete [ 395.176420] Lustre: DEBUG MARKER: == conf-sanity test 76a: set permanent params with lctl across mounts ========================================================== 20:29:38 (1773102578) [ 399.947577] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 400.270330] LustreError: 18815:0:(ldlm_lib.c:1178:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 400.314187] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 403.538241] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing set_default_debug -1 all [ 405.472985] LustreError: 18815:0:(ldlm_lib.c:1178:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 408.879715] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 411.862028] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing set_default_debug -1 all [ 415.473430] Lustre: DEBUG MARKER: oleg147-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 417.902404] Lustre: DEBUG MARKER: oleg147-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 421.917788] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 425.964386] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing set_default_debug -1 all [ 429.759426] Lustre: DEBUG MARKER: oleg147-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 436.494297] Lustre: Modifying parameter general.osc.*.max_dirty_mb=452 in log params [ 443.151986] Lustre: server umount lustre-MDT0000 complete [ 444.896946] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 444.901466] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 444.904401] LustreError: 18812:0:(ldlm_lib.c:1178:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 444.908870] LustreError: Skipped 1 previous similar message [ 444.930128] LustreError: 18812:0:(ldlm_lib.c:1178:target_handle_connect()) Skipped 2 previous similar messages [ 445.660352] LustreError: 18797:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1773102629 with bad export cookie 15142586707670908411 [ 445.661325] LustreError: MGC192.168.201.147@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 445.668770] LustreError: 18797:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 445.871899] Lustre: server umount lustre-MDT0001 complete [ 458.580831] Lustre: server umount lustre-OST0000 complete [ 472.162584] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing load_modules_local [ 479.023994] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 479.299068] LustreError: 22839:0:(ldlm_lib.c:1178:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 479.358621] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 479.361853] Lustre: Skipped 2 previous similar messages [ 482.286511] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing set_default_debug -1 all [ 487.864649] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 490.711189] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing set_default_debug -1 all [ 492.666922] Lustre: Setting parameter general.debug_raw_pointers=Y in log params [ 497.586953] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 501.883991] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing set_default_debug -1 all [ 506.975323] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 507.079548] Lustre: lustre-OST0001: new disk, initializing [ 507.082635] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 508.273760] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 508.285242] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 508.389413] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 510.587707] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing set_default_debug -1 all [ 517.515741] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 519.908868] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 533.986509] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 533.998506] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 534.005195] Lustre: Skipped 5 previous similar messages [ 534.016570] Lustre: Skipped 5 previous similar messages [ 539.585058] Lustre: server umount lustre-MDT0000 complete [ 542.332144] LustreError: 23561:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1773102726 with bad export cookie 15142586707670909258 [ 542.333092] LustreError: MGC192.168.201.147@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 542.340959] LustreError: 23561:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 542.674594] Lustre: server umount lustre-MDT0001 complete [ 556.154982] Lustre: server umount lustre-OST0000 complete [ 569.274243] Lustre: server umount lustre-OST0001 complete [ 581.338680] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing load_modules_local [ 589.314597] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 589.713842] LustreError: 28090:0:(ldlm_lib.c:1178:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 589.729329] LustreError: 28090:0:(ldlm_lib.c:1178:target_handle_connect()) Skipped 1 previous similar message [ 593.065882] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing set_default_debug -1 all [ 599.312405] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 602.423303] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing set_default_debug -1 all [ 604.499790] Lustre: 29203:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 610.131019] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 610.488756] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 610.493247] Lustre: Skipped 5 previous similar messages [ 614.875418] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing set_default_debug -1 all [ 615.596877] LustreError: 29556:0:(ldlm_lib.c:1178:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 615.608291] LustreError: 29556:0:(ldlm_lib.c:1178:target_handle_connect()) Skipped 1 previous similar message [ 620.934824] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 625.442467] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing set_default_debug -1 all [ 631.017166] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 633.531582] Lustre: 31052:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 645.097493] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 645.104449] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 645.118217] Lustre: Skipped 2 previous similar messages [ 645.124714] Lustre: Skipped 7 previous similar messages [ 647.617752] Lustre: server umount lustre-MDT0000 complete [ 649.955180] LustreError: 31054:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1773102834 with bad export cookie 15142586707670910553 [ 649.961892] LustreError: 31054:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 649.964604] LustreError: MGC192.168.201.147@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 650.194055] Lustre: server umount lustre-MDT0001 complete [ 661.828777] Lustre: server umount lustre-OST0000 complete [ 673.951582] Lustre: server umount lustre-OST0001 complete [ 680.483913] Lustre: DEBUG MARKER: == conf-sanity test 76b: verify params log setup correctly ========================================================== 20:34:23 (1773102863) [ 701.452913] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing load_modules_local [ 708.334535] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 708.617026] LustreError: 33840:0:(ldlm_lib.c:1178:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 711.180524] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing set_default_debug -1 all [ 716.397931] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 718.859862] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing set_default_debug -1 all [ 720.496392] Lustre: 34957:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 724.221668] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 727.790552] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing set_default_debug -1 all [ 732.751277] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 736.496664] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing set_default_debug -1 all [ 741.070970] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 743.305390] Lustre: 36806:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 752.609057] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 752.617127] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 752.627485] Lustre: Skipped 1 previous similar message [ 752.630585] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 757.140526] Lustre: server umount lustre-MDT0000 complete [ 759.550368] LustreError: 33821:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1773102943 with bad export cookie 15142586707670911659 [ 759.558044] LustreError: MGC192.168.201.147@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 759.559906] LustreError: 33821:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 759.741643] Lustre: server umount lustre-MDT0001 complete [ 772.273565] Lustre: server umount lustre-OST0000 complete [ 784.801930] Lustre: server umount lustre-OST0001 complete [ 790.313599] Lustre: DEBUG MARKER: == conf-sanity test 76c: verify changelog_mask is applied with lctl set_param -P ========================================================== 20:36:13 (1773102973) [ 800.332233] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing load_modules_local [ 807.549159] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 807.921523] LustreError: 38930:0:(ldlm_lib.c:1178:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 807.939312] LustreError: 38930:0:(ldlm_lib.c:1178:target_handle_connect()) Skipped 3 previous similar messages [ 810.911774] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing set_default_debug -1 all [ 816.678378] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 819.596930] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing set_default_debug -1 all [ 821.698820] Lustre: 40044:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 825.934175] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 829.889170] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing set_default_debug -1 all [ 835.209869] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 839.059208] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing set_default_debug -1 all [ 843.968273] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 851.766929] Lustre: Setting parameter general.mdd.*.changelog_mask=-CLOSE in log params [ 851.771185] Lustre: Skipped 1 previous similar message [ 856.037646] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 856.042396] Lustre: Skipped 7 previous similar messages [ 859.915414] Lustre: server umount lustre-MDT0000 complete [ 862.174044] LustreError: 38911:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1773103046 with bad export cookie 15142586707670912765 [ 862.181041] LustreError: MGC192.168.201.147@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 862.184622] LustreError: 38911:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 868.457743] Lustre: server umount lustre-MDT0001 complete [ 873.844886] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 886.752269] LustreError: 3644:0:(client.c:1390:ptlrpc_import_delay_req()) @@@ invalidate in flight req@ffff9d8e050fc380 x1859232480979328/t0(0) o250->MGC192.168.201.147@tcp@0@lo:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl Rpc:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 887.081728] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 887.086606] Lustre: Skipped 9 previous similar messages [ 889.781711] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing set_default_debug -1 all [ 895.257562] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 897.580350] Lustre: lustre-MDT0000-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 897.677300] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing set_default_debug -1 all [ 900.817261] Lustre: DEBUG MARKER: oleg147-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 901.636206] Lustre: DEBUG MARKER: mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec [ 903.449724] Lustre: DEBUG MARKER: oleg147-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 904.312511] Lustre: DEBUG MARKER: mdc.lustre-MDT0001-mdc-*.mds_server_uuid in FULL state after 0 sec [ 907.748883] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 907.758091] Lustre: Skipped 11 previous similar messages [ 912.945073] Lustre: server umount lustre-MDT0000 complete [ 915.031506] LustreError: 38912:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1773103099 with bad export cookie 15142586707670913955 [ 915.035295] LustreError: MGC192.168.201.147@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 915.040954] LustreError: 38912:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 915.292963] Lustre: server umount lustre-MDT0001 complete [ 927.258334] Lustre: server umount lustre-OST0000 complete [ 939.674474] Lustre: server umount lustre-OST0001 complete [ 944.853530] Lustre: DEBUG MARKER: == conf-sanity test 76d: verify llite.*.xattr_cache can be set by 'lctl set_param -P' correctly ========================================================== 20:38:48 (1773103128) [ 954.276200] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing load_modules_local [ 960.057521] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 960.295533] LustreError: 46264:0:(ldlm_lib.c:1178:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 960.304958] LustreError: 46264:0:(ldlm_lib.c:1178:target_handle_connect()) Skipped 22 previous similar messages [ 962.325820] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing set_default_debug -1 all [ 966.911785] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 969.359249] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing set_default_debug -1 all [ 971.051168] Lustre: 47385:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 971.058941] Lustre: 47385:0:(mgs_llog.c:1348:mgs_modify_param()) Skipped 1 previous similar message [ 975.226426] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 978.492771] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing set_default_debug -1 all [ 982.968900] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 986.370595] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing set_default_debug -1 all [ 990.939351] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 998.329358] Lustre: Setting parameter general.llite.*.xattr_cache=0 in log params [ 1006.053543] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1006.063515] Lustre: Skipped 17 previous similar messages [ 1011.117623] Lustre: server umount lustre-MDT0000 complete [ 1013.023755] LustreError: 49356:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1773103197 with bad export cookie 15142586707670914753 [ 1013.027648] LustreError: MGC192.168.201.147@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1013.037792] LustreError: 49356:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1013.219565] Lustre: server umount lustre-MDT0001 complete [ 1025.543460] Lustre: server umount lustre-OST0000 complete [ 1037.568220] Lustre: server umount lustre-OST0001 complete [ 1042.538611] Lustre: DEBUG MARKER: == conf-sanity test 77: comma-separated MGS NIDs and failover node NIDs ========================================================== 20:40:26 (1773103226) [ 1043.855441] Lustre: DEBUG MARKER: SKIP: conf-sanity test_77 mixed loopback and real device not working [ 1044.704883] Lustre: DEBUG MARKER: == conf-sanity test 78: run resize2fs on MDT and OST filesystems ========================================================== 20:40:28 (1773103228) [ 1050.062909] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing set_hostid [ 1054.488178] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing load_modules_local [ 1066.169665] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1066.323598] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1066.375630] Lustre: lustre-MDT0000: new disk, initializing [ 1066.425150] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1068.505646] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing set_default_debug -1 all [ 1073.403661] Lustre: DEBUG MARKER: oleg147-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1077.753144] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1079.319417] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 1079.340772] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 1081.099442] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing set_default_debug -1 all [ 1085.316802] Lustre: DEBUG MARKER: oleg147-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1086.585228] Lustre: DEBUG MARKER: create test files [ 1104.867268] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1108.738900] Lustre: server umount lustre-OST0000 complete [ 1110.906635] Lustre: server umount lustre-MDT0000 complete [ 1116.385333] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing unload_modules_local [ 1117.870283] Key type lgssc unregistered [ 1118.040293] LNet: 54777:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1118.044442] LNetError: 54777:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1118.055030] LNet: Removed LNI 192.168.201.147@tcp [ 1118.535143] Key type .llcrypt unregistered [ 1118.536895] Key type ._llcrypt unregistered [ 1134.055609] Key type ._llcrypt registered [ 1134.057819] Key type .llcrypt registered [ 1134.128767] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing load_modules_local [ 1134.739568] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1134.747872] alg: No test for adler32 (adler32-zlib) [ 1135.657089] Lustre: Lustre: Build Version: 2.17.50_236_gea5aa0c [ 1135.785294] LNet: Added LNI 192.168.201.147@tcp [8/256/0/180] [ 1137.399171] Key type lgssc registered [ 1137.994919] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1142.065501] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1142.080528] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1143.394036] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1145.571430] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing set_default_debug -1 all [ 1148.566841] Lustre: DEBUG MARKER: oleg147-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1152.166642] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1152.377142] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1153.382215] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:102 to 0x240000400:129) [ 1153.426564] Lustre: 57356:0:(osd_internal.h:1462:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 511 < left 544, rollback = 9 [ 1153.432305] Lustre: 57356:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 32/128/4, destroy: 0/0/0 [ 1153.435559] Lustre: 57356:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 1153.439536] Lustre: 57356:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/13/1, punch: 0/0/0, quota 1/3/0 [ 1153.443182] Lustre: 57356:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 1153.445793] Lustre: 57356:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 1155.134800] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing set_default_debug -1 all [ 1157.707925] Lustre: DEBUG MARKER: oleg147-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1161.290952] Lustre: DEBUG MARKER: check files after expanding the MDT and OST filesystems [ 1164.104691] Lustre: DEBUG MARKER: create more files after expanding the MDT and OST filesystems [ 1168.865064] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1168.865282] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1168.869627] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1172.959035] Lustre: server umount lustre-OST0000 complete [ 1174.874826] Lustre: server umount lustre-MDT0000 complete [ 1179.930728] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing unload_modules_local [ 1181.348193] Key type lgssc unregistered [ 1181.527780] LNet: 58834:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1181.532048] LNetError: 58834:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1181.545705] LNet: Removed LNI 192.168.201.147@tcp [ 1182.028939] Key type .llcrypt unregistered [ 1182.030904] Key type ._llcrypt unregistered [ 1196.740695] Key type ._llcrypt registered [ 1196.742283] Key type .llcrypt registered [ 1196.814285] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing load_modules_local [ 1197.522869] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1197.544906] alg: No test for adler32 (adler32-zlib) [ 1198.441489] Lustre: Lustre: Build Version: 2.17.50_236_gea5aa0c [ 1198.563667] LNet: Added LNI 192.168.201.147@tcp [8/256/0/180] [ 1200.159439] Key type lgssc registered [ 1200.703029] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1204.777432] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1204.791633] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1206.088979] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1208.127667] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing set_default_debug -1 all [ 1210.900132] Lustre: DEBUG MARKER: oleg147-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1214.437429] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1214.587629] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1216.617880] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:140 to 0x240000400:161) [ 1216.657508] Lustre: 61511:0:(osd_internal.h:1462:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 509 < left 544, rollback = 9 [ 1216.663412] Lustre: 61511:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 32/128/6, destroy: 0/0/0 [ 1216.670599] Lustre: 61511:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 1216.675276] Lustre: 61511:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/13/1, punch: 0/0/0, quota 1/3/0 [ 1216.682110] Lustre: 61511:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 1216.688057] Lustre: 61511:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 1217.246401] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing set_default_debug -1 all [ 1219.804474] Lustre: DEBUG MARKER: oleg147-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1223.286529] Lustre: DEBUG MARKER: check files after shrinking the MDT and OST filesystems [ 1227.233130] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1227.233169] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1227.243507] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1232.352485] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1232.815790] Lustre: server umount lustre-OST0000 complete [ 1234.980228] Lustre: server umount lustre-MDT0000 complete [ 1240.080242] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing unload_modules_local [ 1241.630155] Key type lgssc unregistered [ 1241.803530] LNet: 62938:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1241.807552] LNetError: 62938:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1241.816524] LNet: Removed LNI 192.168.201.147@tcp [ 1242.272530] Key type .llcrypt unregistered [ 1242.273934] Key type ._llcrypt unregistered [ 1250.661199] Key type ._llcrypt registered [ 1250.662506] Key type .llcrypt registered [ 1250.716743] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing set_hostid [ 1258.484132] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing load_modules_local [ 1258.878095] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1258.893863] alg: No test for adler32 (adler32-zlib) [ 1259.803465] Lustre: Lustre: Build Version: 2.17.50_236_gea5aa0c [ 1259.927101] LNet: Added LNI 192.168.201.147@tcp [8/256/0/180] [ 1261.527214] Key type lgssc registered [ 1262.089726] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1279.049298] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1279.061340] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1280.225611] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 1280.248303] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1280.294579] Lustre: lustre-MDT0000: new disk, initializing [ 1280.357409] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1280.370574] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1282.568779] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing set_default_debug -1 all [ 1288.601937] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1288.677894] Lustre: 66033:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 1288.734601] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 1288.739186] Lustre: Skipped 1 previous similar message [ 1288.786107] Lustre: lustre-MDT0001: new disk, initializing [ 1288.842086] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 1288.866619] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 1288.873726] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 1290.954437] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing set_default_debug -1 all [ 1295.288453] Lustre: DEBUG MARKER: oleg147-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1297.008552] Lustre: DEBUG MARKER: oleg147-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 1300.305276] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1300.465586] Lustre: lustre-OST0000: new disk, initializing [ 1300.470311] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1300.522044] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1303.293685] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing set_default_debug -1 all [ 1305.113721] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 1305.118864] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 1305.175699] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 1307.358320] Lustre: DEBUG MARKER: oleg147-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1309.333666] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 1309.453831] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 1311.419227] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 1311.523450] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 1315.296990] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1315.298198] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1315.303989] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1315.310216] Lustre: Skipped 1 previous similar message [ 1318.653744] Lustre: server umount lustre-OST0000 complete [ 1325.025872] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1325.029203] LustreError: Skipped 1 previous similar message [ 1325.031327] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1325.037990] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1325.041027] Lustre: Skipped 1 previous similar message [ 1326.637666] Lustre: server umount lustre-MDT0000 complete [ 1328.394080] LustreError: 66810:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1773103512 with bad export cookie 15123164750442876230 [ 1328.398811] LustreError: MGC192.168.201.147@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1328.401589] LustreError: 66810:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1328.559841] Lustre: server umount lustre-MDT0001 complete [ 1332.716648] Lustre: DEBUG MARKER: == conf-sanity test 79: format MDT/OST without mgs option (should return errors) ========================================================== 20:45:16 (1773103516) [ 1337.995285] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing load_modules_local [ 1357.631124] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing set_hostid [ 1362.292842] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing load_modules_local [ 1380.536824] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1380.673981] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 1380.697734] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1380.750579] Lustre: lustre-MDT0000: new disk, initializing [ 1380.808412] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1380.822674] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1382.775963] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing set_default_debug -1 all [ 1388.322395] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1388.378704] Lustre: 73586:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 1388.411754] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 1388.414670] Lustre: Skipped 1 previous similar message [ 1388.457723] Lustre: lustre-MDT0001: new disk, initializing [ 1388.499806] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 1388.517806] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 1388.523740] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 1390.371192] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing set_default_debug -1 all [ 1394.506743] Lustre: DEBUG MARKER: oleg147-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1396.299691] Lustre: DEBUG MARKER: oleg147-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 1399.698609] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1399.823888] Lustre: lustre-OST0000: new disk, initializing [ 1399.826737] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1399.862405] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1401.280274] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 1401.285496] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 1401.331810] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 1402.524446] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing set_default_debug -1 all [ 1406.454761] Lustre: DEBUG MARKER: oleg147-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1408.386777] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 1408.486254] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 1410.325615] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 1410.431239] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 1411.554798] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 1411.554896] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1411.556555] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1411.556562] Lustre: Skipped 1 previous similar message [ 1411.558374] LustreError: Skipped 1 previous similar message [ 1411.564864] Lustre: Skipped 2 previous similar messages [ 1416.674937] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1416.679837] Lustre: Skipped 2 previous similar messages [ 1417.447844] Lustre: server umount lustre-OST0000 complete [ 1419.232180] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1419.236320] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1421.793164] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1421.796285] Lustre: Skipped 2 previous similar messages [ 1425.266986] Lustre: server umount lustre-MDT0000 complete [ 1427.061292] LustreError: 73577:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1773103611 with bad export cookie 15123164750442877119 [ 1427.065748] LustreError: MGC192.168.201.147@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1427.067085] LustreError: 73577:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1427.235829] Lustre: server umount lustre-MDT0001 complete [ 1430.981726] Lustre: DEBUG MARKER: == conf-sanity test 80: mgc import reconnect race ======== 20:46:54 (1773103614) [ 1434.097746] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1434.295127] LustreError: 77588:0:(ldlm_lib.c:1178:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1434.304180] LustreError: 77588:0:(ldlm_lib.c:1178:target_handle_connect()) Skipped 3 previous similar messages [ 1434.324844] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1436.038755] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing set_default_debug -1 all [ 1439.712063] LustreError: 77589:0:(ldlm_lib.c:1178:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1439.938647] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1442.079421] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing set_default_debug -1 all [ 1444.708850] Lustre: DEBUG MARKER: oleg147-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1446.473631] Lustre: DEBUG MARKER: oleg147-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 1449.679544] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1452.605954] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing set_default_debug -1 all [ 1455.264079] Lustre: DEBUG MARKER: oleg147-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1457.036399] Lustre: 79917:0:(genops.c:1793:obd_export_evict_by_uuid()) MGS: evicting 6d3b4060-5c05-4c36-a4f7-f6f10ab3c947 at adminstrative request [ 1459.042366] LustreError: MGC192.168.201.147@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1459.051341] Lustre: *** cfs_fail_loc=906, val=2147483648*** [ 1469.411692] Lustre: MGS: Client 6d3b4060-5c05-4c36-a4f7-f6f10ab3c947 (at 0@lo) reconnecting [ 1469.418137] Lustre: Evicted from MGS (at 0@lo) after server handle changed from 0xd1e04636eddb5431 to 0xd1e04636eddb56a7 [ 1469.423020] Lustre: MGC192.168.201.147@tcp: Connection restored to 0@lo (at 0@lo) [ 1490.581508] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1490.681171] Lustre: lustre-OST0001: new disk, initializing [ 1490.684291] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 1490.721892] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 1490.727602] Lustre: Skipped 2 previous similar messages [ 1493.459222] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing set_default_debug -1 all [ 1497.353711] Lustre: DEBUG MARKER: oleg147-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid [ 1498.672229] Lustre: server umount lustre-OST0001 complete [ 1500.662628] LustreError: 79182:0:(ldlm_lib.c:1178:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1500.663201] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1500.675792] Lustre: Skipped 1 previous similar message [ 1500.679564] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1505.247905] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1505.764822] LustreError: 79956:0:(ldlm_lib.c:1178:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1505.772233] LustreError: 79956:0:(ldlm_lib.c:1178:target_handle_connect()) Skipped 2 previous similar messages [ 1506.519461] Lustre: server umount lustre-OST0000 complete [ 1510.883575] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1510.890351] Lustre: Skipped 2 previous similar messages [ 1514.486175] Lustre: server umount lustre-MDT0000 complete [ 1515.999967] LustreError: 77589:0:(ldlm_lib.c:1178:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1516.005657] LustreError: 77589:0:(ldlm_lib.c:1178:target_handle_connect()) Skipped 1 previous similar message [ 1516.277691] LustreError: 79939:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1773103700 with bad export cookie 15123164750442878631 [ 1516.282279] LustreError: MGC192.168.201.147@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1516.283755] LustreError: 79939:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1516.445123] Lustre: server umount lustre-MDT0001 complete [ 1520.606351] Lustre: DEBUG MARKER: == conf-sanity test 81: sparse OST indexing ============== 20:48:24 (1773103704) [ 1521.348302] Lustre: DEBUG MARKER: SKIP: conf-sanity test_81 needs >= 3 OSTs [ 1522.182457] Lustre: DEBUG MARKER: == conf-sanity test 82a: specify OSTs for file (succeed) or directory (succeed) ========================================================== 20:48:26 (1773103706) [ 1522.944273] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82a needs >= 3 OSTs [ 1523.767645] Lustre: DEBUG MARKER: == conf-sanity test 82b: specify OSTs for file with --pool and --ost-list options ========================================================== 20:48:27 (1773103707) [ 1524.476685] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82b needs >= 4 OSTs [ 1525.286669] Lustre: DEBUG MARKER: == conf-sanity test 82c: specify sparse OSTs for setstripe (should not crash) ========================================================== 20:48:29 (1773103709) [ 1525.992490] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82c needs >= 4 OSTs [ 1526.795971] Lustre: DEBUG MARKER: == conf-sanity test 83: ENOSPACE on OST doesn't cause message VFS: Busy inodes after unmount ... ========================================================== 20:48:30 (1773103710) [ 1527.539886] Lustre: DEBUG MARKER: mount the OST /dev/mapper/ost1_flakey as a ldiskfs filesystem [ 1530.266583] loop: module loaded [ 1530.272274] loop0: detected capacity change from 0 to 4294967296 [ 1530.296627] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 1531.007898] Lustre: DEBUG MARKER: run llverfs in partial mode on the OST ldiskfs /mnt/lustre-ost1 [ 1532.385607] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing run_llverfs /mnt/lustre-ost1 -vpl no [ 1534.435316] Lustre: DEBUG MARKER: unmount the OST /dev/mapper/ost1_flakey [ 1546.918983] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1546.940307] LustreError: 84296:0:(obd_config.c:880:class_setup()) setup lustre-OST0000-osd failed (-28) [ 1546.945354] LustreError: 84296:0:(obd_mount.c:217:lustre_start_simple()) lustre-OST0000-osd setup error -28 [ 1546.950080] LustreError: 84296:0:(tgt_mount.c:2534:server_fill_super()) Unable to start osd on /dev/mapper/ost1_flakey: -28 [ 1546.954802] LustreError: 84296:0:(super25.c:186:lustre_fill_super()) llite: Unable to mount : rc = -28 [ 1555.688138] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing set_hostid [ 1559.893148] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing load_modules_local [ 1576.879430] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1577.006552] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 1577.022157] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1577.069294] Lustre: lustre-MDT0000: new disk, initializing [ 1577.116296] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1577.127455] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1578.960885] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing set_default_debug -1 all [ 1584.348251] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1584.398515] Lustre: 87123:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 1584.539687] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 1586.283825] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing set_default_debug -1 all [ 1590.143900] Lustre: DEBUG MARKER: oleg147-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1591.708487] Lustre: DEBUG MARKER: oleg147-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 1594.606930] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1597.274740] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing set_default_debug -1 all [ 1601.216153] Lustre: DEBUG MARKER: oleg147-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1601.541739] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 1601.590053] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 1603.082848] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 1603.175469] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 1604.990514] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 1605.100109] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 1606.625996] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1606.627341] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1606.632350] LustreError: Skipped 1 previous similar message [ 1606.633241] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1606.637045] Lustre: Skipped 1 previous similar message [ 1606.638472] Lustre: Skipped 4 previous similar messages [ 1608.047895] Lustre: server umount lustre-OST0000 complete [ 1615.981134] Lustre: server umount lustre-MDT0000 complete [ 1616.865573] LustreError: 87135:0:(ldlm_lib.c:1178:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1616.872357] LustreError: 87135:0:(ldlm_lib.c:1178:target_handle_connect()) Skipped 1 previous similar message [ 1617.754801] LustreError: 88865:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1773103802 with bad export cookie 15123164750442879212 [ 1617.756682] LustreError: MGC192.168.201.147@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1617.761187] LustreError: 88865:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 1617.914835] Lustre: server umount lustre-MDT0001 complete [ 1621.976569] Lustre: DEBUG MARKER: == conf-sanity test 84: check recovery_hard_time ========= 20:50:05 (1773103805) [ 1625.256576] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1627.355823] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing set_default_debug -1 all [ 1631.211318] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1633.035729] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing set_default_debug -1 all [ 1635.416954] Lustre: DEBUG MARKER: oleg147-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1637.007585] Lustre: DEBUG MARKER: oleg147-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 1640.156565] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1642.799324] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing set_default_debug -1 all [ 1645.061353] Lustre: DEBUG MARKER: oleg147-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1646.712370] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 1650.904988] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 4 sec [ 1652.687841] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 1652.792355] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 1655.925673] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1656.013196] Lustre: lustre-OST0001: new disk, initializing [ 1656.015506] Lustre: Skipped 2 previous similar messages [ 1656.018839] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 1656.026597] Lustre: Skipped 3 previous similar messages [ 1658.524787] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing set_default_debug -1 all [ 1662.352758] Lustre: DEBUG MARKER: oleg147-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid [ 1664.025431] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 1664.030393] Lustre: Skipped 2 previous similar messages [ 1664.034263] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 1664.073765] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 1664.182301] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid 50 [ 1664.284481] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 1666.020985] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [ 1666.114896] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 1677.714210] Lustre: DEBUG MARKER: mds1 REPLAY BARRIER on lustre-MDT0000 [ 1687.093840] Lustre: Failing over lustre-MDT0000 [ 1687.547909] Lustre: server umount lustre-MDT0000 complete [ 1688.032784] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1688.037823] LustreError: Skipped 1 previous similar message [ 1688.040059] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1688.049624] Lustre: Skipped 2 previous similar messages [ 1688.055026] LustreError: 91839:0:(ldlm_lib.c:1178:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1688.063334] LustreError: 91839:0:(ldlm_lib.c:1178:target_handle_connect()) Skipped 2 previous similar messages [ 1704.055719] LDISKFS-fs (dm-0): 4 truncates cleaned up [ 1704.057928] LDISKFS-fs (dm-0): recovery complete [ 1704.063158] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1704.125921] LustreError: MGC192.168.201.147@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1704.293544] Lustre: lustre-MDT0000: in recovery but waiting for the first client to connect [ 1706.079811] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing set_default_debug -1 all [ 1708.583911] Lustre: lustre-MDT0000: Will be in recovery for at least 1:00, or until 3 clients reconnect [ 1708.584557] Lustre: 91123:0:(ldlm_lib.c:2067:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 1709.537225] Lustre: 91128:0:(ldlm_lib.c:2067:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 1709.545873] Lustre: 91128:0:(ldlm_lib.c:2067:extend_recovery_timer()) Skipped 1 previous similar message [ 1709.553461] Lustre: lustre-MDT0000-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 1709.594376] LustreError: 96838:0:(ldlm_lib.c:2680:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 1709.911172] LustreError: 96838:0:(ldlm_lib.c:2680:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 1710.234358] LustreError: 96838:0:(ldlm_lib.c:2680:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 1710.243418] LustreError: 96838:0:(ldlm_lib.c:2680:replay_request_or_update()) Skipped 1 previous similar message [ 1710.559163] LustreError: 96838:0:(ldlm_lib.c:2680:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 1710.562593] LustreError: 96838:0:(ldlm_lib.c:2680:replay_request_or_update()) Skipped 1 previous similar message [ 1710.568387] Lustre: 96838:0:(ldlm_lib.c:2067:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 1710.575250] Lustre: 96838:0:(ldlm_lib.c:2067:extend_recovery_timer()) Skipped 7 previous similar messages [ 1711.521709] LustreError: 96838:0:(ldlm_lib.c:2680:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 1711.527503] LustreError: 96838:0:(ldlm_lib.c:2680:replay_request_or_update()) Skipped 3 previous similar messages [ 1711.847105] LustreError: 96838:0:(ldlm_lib.c:2680:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 1711.850777] LustreError: 96838:0:(ldlm_lib.c:2680:replay_request_or_update()) Skipped 3 previous similar messages [ 1712.794880] Lustre: 96838:0:(ldlm_lib.c:2067:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 1712.800975] Lustre: 96838:0:(ldlm_lib.c:2067:extend_recovery_timer()) Skipped 6 previous similar messages [ 1713.744448] LustreError: 96838:0:(ldlm_lib.c:2680:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 1713.749754] LustreError: 96838:0:(ldlm_lib.c:2680:replay_request_or_update()) Skipped 6 previous similar messages [ 1714.063351] LustreError: 96838:0:(ldlm_lib.c:2680:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 1714.067386] LustreError: 96838:0:(ldlm_lib.c:2680:replay_request_or_update()) Skipped 6 previous similar messages [ 1716.905772] Lustre: 96838:0:(ldlm_lib.c:2067:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 1716.913287] Lustre: 96838:0:(ldlm_lib.c:2067:extend_recovery_timer()) Skipped 12 previous similar messages [ 1717.865106] LustreError: 96838:0:(ldlm_lib.c:2680:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 1717.872813] LustreError: 96838:0:(ldlm_lib.c:2680:replay_request_or_update()) Skipped 12 previous similar messages [ 1718.191312] LustreError: 96838:0:(ldlm_lib.c:2680:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 1718.194803] LustreError: 96838:0:(ldlm_lib.c:2680:replay_request_or_update()) Skipped 12 previous similar messages [ 1725.088436] Lustre: 96838:0:(ldlm_lib.c:2067:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 1725.095354] Lustre: 96838:0:(ldlm_lib.c:2067:extend_recovery_timer()) Skipped 25 previous similar messages [ 1726.041717] LustreError: 96838:0:(ldlm_lib.c:2680:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 1726.046992] LustreError: 96838:0:(ldlm_lib.c:2680:replay_request_or_update()) Skipped 25 previous similar messages [ 1726.367131] LustreError: 96838:0:(ldlm_lib.c:2680:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 1726.369713] LustreError: 96838:0:(ldlm_lib.c:2680:replay_request_or_update()) Skipped 25 previous similar messages [ 1741.090202] Lustre: 96838:0:(ldlm_lib.c:2067:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 1741.098319] Lustre: 96838:0:(ldlm_lib.c:2067:extend_recovery_timer()) Skipped 50 previous similar messages [ 1742.353686] LustreError: 96838:0:(ldlm_lib.c:2680:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 1742.359728] LustreError: 96838:0:(ldlm_lib.c:2680:replay_request_or_update()) Skipped 51 previous similar messages [ 1742.679299] LustreError: 96838:0:(ldlm_lib.c:2680:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 1742.683699] LustreError: 96838:0:(ldlm_lib.c:2680:replay_request_or_update()) Skipped 51 previous similar messages [ 1768.287145] Lustre: 96838:0:(ldlm_lib.c:2376:target_recovery_overseer()) lustre-MDT0000 recovery is aborted by hard timeout [ 1768.291283] Lustre: 96838:0:(ldlm_lib.c:2386:target_recovery_overseer()) recovery is aborted, evict exports in recovery [ 1768.296240] Lustre: 96838:0:(genops.c:1620:class_disconnect_stale_exports()) lustre-MDT0000: disconnect stale client 77e4bfa9-f5e4-4681-9c34-8434da219db4@192.168.201.47@tcp [ 1768.301862] Lustre: lustre-MDT0000: disconnecting 1 stale clients [ 1768.304240] Lustre: 96838:0:(ldlm_lib.c:1896:abort_req_replay_queue()) @@@ aborted: req@00000000549ee869 x1859233708578048/t0(8589934784) o35->77e4bfa9-f5e4-4681-9c34-8434da219db4@192.168.201.47@tcp:53/0 lens 392/0 e 0 to 0 dl 1773103963 ref 1 fl Complete:/604/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:0 [ 1768.320178] Lustre: lustre-MDT0000-osd: cancel update llog [0x200000400:0x1:0x0] [ 1768.332054] Lustre: lustre-MDT0000: Denying connection for new client 77e4bfa9-f5e4-4681-9c34-8434da219db4 (at 192.168.201.47@tcp), waiting for 3 known clients (2 recovered, 0 in progress, and 1 evicted) to recover in 0:00 [ 1768.336611] Lustre: lustre-MDT0001-osp-MDT0000: cancel update llog [0x240000401:0x1:0x0] [ 1768.355883] Lustre: lustre-MDT0000-osp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 1768.359758] Lustre: Skipped 2 previous similar messages [ 1768.360671] Lustre: lustre-MDT0000: Recovery over after 1:00, of 3 clients 2 recovered and 1 was evicted. [ 1768.383060] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:81 to 0x280000401:577) [ 1768.383065] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:49 to 0x2c0000401:545) [ 1774.050943] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1774.054869] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1774.060080] Lustre: Skipped 3 previous similar messages [ 1774.062860] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1774.070690] Lustre: Skipped 3 previous similar messages [ 1779.783072] Lustre: server umount lustre-OST0000 complete [ 1781.222847] LustreError: 92722:0:(ldlm_lib.c:1178:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1781.231051] LustreError: 92722:0:(ldlm_lib.c:1178:target_handle_connect()) Skipped 21 previous similar messages [ 1787.550327] Lustre: server umount lustre-OST0001 complete [ 1795.382726] Lustre: server umount lustre-MDT0000 complete [ 1797.008847] LustreError: 91847:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1773103981 with bad export cookie 15123164750442930410 [ 1797.012653] LustreError: MGC192.168.201.147@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1797.014067] LustreError: 91847:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1797.161404] Lustre: server umount lustre-MDT0001 complete [ 1800.943053] Lustre: DEBUG MARKER: == conf-sanity test 85: osd_ost init: fail ea_fid_set ==== 20:53:04 (1773103984) [ 1804.198415] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1804.202157] Lustre: *** cfs_fail_loc=197, val=0*** [ 1819.679628] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 1824.799696] LustreError: 98796:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.201.147@tcp: failed processing log, type 4: rc = -110 [ 1829.927120] Lustre: *** cfs_fail_loc=197, val=0*** [ 1829.928797] Lustre: Skipped 65 previous similar messages [ 1850.399646] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1850.402229] Lustre: Skipped 7 previous similar messages [ 1852.860313] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing set_default_debug -1 all [ 1855.217481] Lustre: DEBUG MARKER: oleg147-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1856.144951] Lustre: server umount lustre-OST0000 complete [ 1859.770778] Lustre: DEBUG MARKER: == conf-sanity test 86: Replacing mkfs.lustre -G option == 20:54:03 (1773104043) [ 1874.636875] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1874.837959] LustreError: 100300:0:(ldlm_lib.c:1178:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1874.850303] LustreError: 100300:0:(ldlm_lib.c:1178:target_handle_connect()) Skipped 4 previous similar messages [ 1876.523075] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing set_default_debug -1 all [ 1880.149417] Lustre: DEBUG MARKER: oleg147-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1882.430509] LustreError: 100333:0:(osp_object.c:617:osp_attr_get()) lustre-MDT0001-osp-MDT0000: osp_attr_get update error [0x200000009:0x1:0x0]: rc = -5 [ 1882.439273] LustreError: 100333:0:(lod_sub_object.c:917:lod_sub_prep_llog()) lustre-MDT0000-mdtlov: can't get id from catalogs: rc = -5 [ 1882.449049] LustreError: 100333:0:(lod_dev.c:510:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 7, retries 0, failed: rc = -5 [ 1882.597977] Lustre: server umount lustre-MDT0000 complete [ 1896.537186] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing set_hostid [ 1900.983497] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing load_modules_local [ 1916.959205] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1917.086061] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 1917.108669] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1917.157872] Lustre: lustre-MDT0000: new disk, initializing [ 1917.200029] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1919.086887] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing set_default_debug -1 all [ 1924.608947] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1924.671074] Lustre: 104352:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 1924.835807] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 1926.786598] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing set_default_debug -1 all [ 1930.596507] Lustre: DEBUG MARKER: oleg147-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1932.252118] Lustre: DEBUG MARKER: oleg147-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 1935.431361] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1938.031419] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing set_default_debug -1 all [ 1941.034619] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 1941.643705] Lustre: DEBUG MARKER: oleg147-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1943.453948] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 1943.550500] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 1945.263211] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 1945.357467] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 1951.202132] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1951.202558] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 1951.204941] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1951.204947] Lustre: Skipped 9 previous similar messages [ 1951.206965] Lustre: Skipped 6 previous similar messages [ 1951.209389] LustreError: Skipped 4 previous similar messages [ 1952.231459] Lustre: server umount lustre-OST0000 complete [ 1960.053124] Lustre: server umount lustre-MDT0000 complete [ 1961.725369] LustreError: 104345:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1773104146 with bad export cookie 15123164750442941092 [ 1961.728381] LustreError: MGC192.168.201.147@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1961.729977] LustreError: 104345:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 1961.888765] Lustre: server umount lustre-MDT0001 complete [ 1965.783310] Lustre: DEBUG MARKER: == conf-sanity test 87: check if MDT inode can hold EAs with N stripes properly ========================================================== 20:55:49 (1773104149) [ 1969.729619] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing unload_modules_local [ 1971.199532] Key type lgssc unregistered [ 1971.356651] LNet: 108415:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1971.359758] LNetError: 108415:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1971.371928] LNet: Removed LNI 192.168.201.147@tcp [ 1971.780437] Key type .llcrypt unregistered [ 1971.783051] Key type ._llcrypt unregistered [ 1979.512868] Key type ._llcrypt registered [ 1979.514583] Key type .llcrypt registered [ 1979.564972] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing set_hostid [ 1987.277482] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing load_modules_local [ 1987.750061] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1987.800512] alg: No test for adler32 (adler32-zlib) [ 1988.721273] Lustre: Lustre: Build Version: 2.17.50_236_gea5aa0c [ 1988.839461] LNet: Added LNI 192.168.201.147@tcp [8/256/0/180] [ 1990.455155] Key type lgssc registered [ 1990.984977] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2009.537188] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 2009.550930] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2010.674728] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2010.691310] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2010.732963] Lustre: lustre-MDT0000: new disk, initializing [ 2010.778986] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2010.789470] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2012.607066] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing set_default_debug -1 all [ 2017.915643] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2018.061997] Lustre: lustre-OST0000: new disk, initializing [ 2018.064642] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 2018.068415] Lustre: Skipped 1 previous similar message [ 2018.109296] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2020.698247] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing set_default_debug -1 all [ 2023.441985] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 2023.447370] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 2023.473841] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 2026.287402] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2026.375524] Lustre: lustre-OST0001: new disk, initializing [ 2026.378582] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 2026.417409] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 2029.041079] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing set_default_debug -1 all [ 2030.482148] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:1:ost [ 2030.486923] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:1:ost] [ 2030.502956] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x280000400 [ 2035.680836] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2035.681885] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2035.686316] Lustre: Skipped 1 previous similar message [ 2039.947413] Lustre: server umount lustre-MDT0000 complete [ 2042.973037] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: (null) [ 2045.118512] LustreError: 111718:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1773104229 with bad export cookie 4303098098581798057 [ 2045.122681] LustreError: MGC192.168.201.147@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2045.127046] LustreError: 111718:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 2045.176929] Lustre: server umount lustre-OST0000 complete [ 2047.010177] Lustre: server umount lustre-OST0001 complete [ 2050.800588] Lustre: DEBUG MARKER: == conf-sanity test 88: check the default mount options can be overridden ========================================================== 20:57:14 (1773104234) [ 2054.972866] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing load_modules_local [ 2065.496511] Lustre: DEBUG MARKER: == conf-sanity test 89: check tunefs --param and --erase-param{s} options ========================================================== 20:57:29 (1773104249) [ 2082.823606] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing set_hostid [ 2087.276726] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing load_modules_local [ 2103.435306] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2103.569128] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2103.585836] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2103.632826] Lustre: lustre-MDT0000: new disk, initializing [ 2103.669569] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2103.679388] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2105.414569] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing set_default_debug -1 all [ 2110.645084] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2110.706226] Lustre: 120034:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2110.742575] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 2110.745168] Lustre: Skipped 1 previous similar message [ 2110.789952] Lustre: lustre-MDT0001: new disk, initializing [ 2110.836101] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2110.852472] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 2110.857128] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2112.693380] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing set_default_debug -1 all [ 2116.539680] Lustre: DEBUG MARKER: oleg147-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2118.157660] Lustre: DEBUG MARKER: oleg147-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2121.303793] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2121.438827] Lustre: lustre-OST0000: new disk, initializing [ 2121.441576] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 2121.494730] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2124.247627] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing set_default_debug -1 all [ 2128.220127] Lustre: DEBUG MARKER: oleg147-client.virtnet: executing wait_import_state_mount (FULL|IDLE) failover.node=192.0.2.254@tcp0,osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2129.925872] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 2129.933642] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2129.989250] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2130.137317] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2130.249916] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2132.098973] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2132.192599] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2135.012369] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2135.012868] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2135.016272] LustreError: Skipped 1 previous similar message [ 2135.023291] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2135.033516] Lustre: Skipped 1 previous similar message [ 2139.237887] Lustre: server umount lustre-OST0000 complete [ 2141.663727] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2141.669067] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2141.676164] Lustre: Skipped 1 previous similar message [ 2141.679502] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2145.249290] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2145.249927] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2147.061188] Lustre: server umount lustre-MDT0000 complete [ 2148.860597] LustreError: 120027:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1773104333 with bad export cookie 4303098098581799282 [ 2148.863301] LustreError: MGC192.168.201.147@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2148.866648] LustreError: 120027:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 2149.025078] Lustre: server umount lustre-MDT0001 complete [ 2152.882281] Lustre: DEBUG MARKER: == conf-sanity test 90a: check max_mod_rpcs_in_flight is enforced ========================================================== 20:58:56 (1773104336) [ 2155.946240] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2156.192352] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2157.968635] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing set_default_debug -1 all [ 2161.632818] LustreError: 124037:0:(ldlm_lib.c:1178:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2161.639861] LustreError: 124037:0:(ldlm_lib.c:1178:target_handle_connect()) Skipped 2 previous similar messages [ 2161.710031] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2163.623278] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing set_default_debug -1 all [ 2166.135657] Lustre: DEBUG MARKER: oleg147-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2167.720839] Lustre: DEBUG MARKER: oleg147-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2170.643133] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2173.218481] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing set_default_debug -1 all [ 2175.654129] Lustre: DEBUG MARKER: oleg147-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2183.399071] Lustre: *** cfs_fail_loc=159, val=0*** [ 2198.564660] Lustre: lustre-MDT0000: Client df50f0db-6967-4ffc-bed3-aec4f93cbc1a (at 192.168.201.47@tcp) reconnecting [ 2199.268180] Lustre: *** cfs_fail_loc=159, val=0*** [ 2199.274707] Lustre: Skipped 5 previous similar messages [ 2215.479200] Lustre: lustre-MDT0000: Client df50f0db-6967-4ffc-bed3-aec4f93cbc1a (at 192.168.201.47@tcp) reconnecting [ 2232.288498] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2232.291107] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2232.292580] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2232.292586] Lustre: Skipped 1 previous similar message [ 2232.300649] Lustre: Skipped 1 previous similar message [ 2236.695579] Lustre: server umount lustre-OST0000 complete [ 2238.946180] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2242.528703] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2242.529030] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2242.531197] Lustre: Skipped 2 previous similar messages [ 2242.535488] Lustre: Skipped 1 previous similar message [ 2244.395914] Lustre: server umount lustre-MDT0000 complete [ 2245.966643] LustreError: 126422:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1773104430 with bad export cookie 4303098098581800150 [ 2245.970149] LustreError: MGC192.168.201.147@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2245.971561] LustreError: 126422:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2246.097574] Lustre: server umount lustre-MDT0001 complete [ 2250.715133] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing unload_modules_local [ 2252.135519] Key type lgssc unregistered [ 2252.306044] LNet: 127461:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2252.309816] LNetError: 127461:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2252.320592] LNet: Removed LNI 192.168.201.147@tcp [ 2252.749498] Key type .llcrypt unregistered [ 2252.751670] Key type ._llcrypt unregistered [ 2263.857518] Key type ._llcrypt registered [ 2263.859550] Key type .llcrypt registered [ 2263.919383] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing load_modules_local [ 2264.611559] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2264.638824] alg: No test for adler32 (adler32-zlib) [ 2265.528224] Lustre: Lustre: Build Version: 2.17.50_236_gea5aa0c [ 2265.648775] LNet: Added LNI 192.168.201.147@tcp [8/256/0/180] [ 2267.255170] Key type lgssc registered [ 2267.780109] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2271.111625] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 2271.120784] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2272.335884] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2273.969406] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing set_default_debug -1 all [ 2277.858315] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2278.083233] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2279.910730] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing set_default_debug -1 all [ 2282.239151] Lustre: DEBUG MARKER: oleg147-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2283.902532] Lustre: DEBUG MARKER: oleg147-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2286.765034] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2286.946749] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2289.540774] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing set_default_debug -1 all [ 2292.050481] Lustre: DEBUG MARKER: oleg147-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2297.191329] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:42 to 0x280000401:65) [ 2303.239272] Lustre: *** cfs_fail_loc=159, val=0*** [ 2319.415239] Lustre: lustre-MDT0000: Client b8c9c832-a8ea-4c1c-b715-9e8efaeb1e97 (at 192.168.201.47@tcp) reconnecting [ 2320.391979] Lustre: *** cfs_fail_loc=159, val=0*** [ 2335.800370] Lustre: lustre-MDT0001: Client b8c9c832-a8ea-4c1c-b715-9e8efaeb1e97 (at 192.168.201.47@tcp) reconnecting [ 2336.488261] Lustre: *** cfs_fail_loc=159, val=0*** [ 2336.491217] Lustre: Skipped 5 previous similar messages [ 2352.689979] Lustre: lustre-MDT0001: Client b8c9c832-a8ea-4c1c-b715-9e8efaeb1e97 (at 192.168.201.47@tcp) reconnecting [ 2367.334256] Lustre: 131575:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 2368.375196] Lustre: *** cfs_fail_loc=159, val=0*** [ 2368.377819] Lustre: Skipped 1 previous similar message [ 2383.909958] Lustre: lustre-MDT0000: Client e65b89a8-d503-41fc-badf-27969b5e8358 (at 192.168.201.47@tcp) reconnecting [ 2384.595201] Lustre: *** cfs_fail_loc=159, val=0*** [ 2384.597635] Lustre: Skipped 14 previous similar messages [ 2400.825460] Lustre: lustre-MDT0000: Client e65b89a8-d503-41fc-badf-27969b5e8358 (at 192.168.201.47@tcp) reconnecting [ 2401.491168] Lustre: 131865:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 2420.193102] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2420.195480] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2420.197592] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2420.204344] Lustre: Skipped 1 previous similar message [ 2424.547971] Lustre: server umount lustre-OST0000 complete [ 2426.848074] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2426.851137] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2426.857692] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2426.860487] Lustre: Skipped 1 previous similar message [ 2430.431961] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2430.433063] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2432.424538] Lustre: server umount lustre-MDT0000 complete [ 2433.873169] LustreError: 128817:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1773104618 with bad export cookie 12104697217933786801 [ 2433.876570] LustreError: MGC192.168.201.147@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2433.877611] LustreError: 128817:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2434.010561] Lustre: server umount lustre-MDT0001 complete [ 2438.205864] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing unload_modules_local [ 2439.462618] Key type lgssc unregistered [ 2439.619525] LNet: 132925:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2439.622362] LNetError: 132925:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2439.630588] LNet: Removed LNI 192.168.201.147@tcp [ 2440.009132] Key type .llcrypt unregistered [ 2440.011269] Key type ._llcrypt unregistered [ 2450.993864] Key type ._llcrypt registered [ 2450.995669] Key type .llcrypt registered [ 2451.040371] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing load_modules_local [ 2451.486102] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2451.540877] alg: No test for adler32 (adler32-zlib) [ 2452.426304] Lustre: Lustre: Build Version: 2.17.50_236_gea5aa0c [ 2452.542746] LNet: Added LNI 192.168.201.147@tcp [8/256/0/180] [ 2454.151167] Key type lgssc registered [ 2454.619039] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2457.712017] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 2457.720205] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2458.910902] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2460.488220] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing set_default_debug -1 all [ 2463.846409] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2463.998118] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2465.683818] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing set_default_debug -1 all [ 2467.905785] Lustre: DEBUG MARKER: oleg147-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2469.348405] Lustre: DEBUG MARKER: oleg147-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2471.989761] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2472.128968] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2474.218495] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:97) [ 2474.381898] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing set_default_debug -1 all [ 2476.557958] Lustre: DEBUG MARKER: oleg147-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2484.457058] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:129) [ 2489.798632] Lustre: 136565:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 2490.625092] Lustre: 136614:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 2491.207801] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 in log params [ 2492.618812] Lustre: 136761:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre/mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 (mode = 0) failed: rc = -17 [ 2492.990696] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=7 in log params [ 2492.993691] Lustre: Skipped 1 previous similar message [ 2493.371703] Lustre: 136859:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 2494.944963] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2494.948178] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2494.949345] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2494.952772] Lustre: Skipped 1 previous similar message [ 2496.669408] Lustre: server umount lustre-OST0000 complete [ 2500.065168] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2500.065264] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2500.073726] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2500.077324] LustreError: Skipped 1 previous similar message [ 2500.081192] Lustre: Skipped 2 previous similar messages [ 2500.393749] Lustre: server umount lustre-MDT0000 complete [ 2501.895438] LustreError: 135915:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1773104686 with bad export cookie 9889376492601995132 [ 2501.897612] LustreError: MGC192.168.201.147@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2501.902138] LustreError: 135915:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 2501.907231] LustreError: 134291:0:(mgc_request.c:614:do_requeue()) failed processing log: -5 [ 2502.062482] Lustre: server umount lustre-MDT0001 complete [ 2506.416236] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing unload_modules_local [ 2507.644862] Key type lgssc unregistered [ 2507.801346] LNet: 137889:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2507.811381] LNetError: 137889:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2507.823372] LNet: Removed LNI 192.168.201.147@tcp [ 2508.184311] Key type .llcrypt unregistered [ 2508.185927] Key type ._llcrypt unregistered [ 2518.873359] Key type ._llcrypt registered [ 2518.874550] Key type .llcrypt registered [ 2518.928642] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing load_modules_local [ 2519.442709] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2519.457079] alg: No test for adler32 (adler32-zlib) [ 2520.377657] Lustre: Lustre: Build Version: 2.17.50_236_gea5aa0c [ 2520.482472] LNet: Added LNI 192.168.201.147@tcp [8/256/0/180] [ 2522.079141] Key type lgssc registered [ 2522.517088] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2525.698822] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 2525.706505] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2526.917264] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2528.593774] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing set_default_debug -1 all [ 2532.106678] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2532.270345] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2533.996164] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing set_default_debug -1 all [ 2536.314573] Lustre: DEBUG MARKER: oleg147-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2537.853344] Lustre: DEBUG MARKER: oleg147-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2540.803289] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2540.983318] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2541.999549] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:129) [ 2543.393835] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing set_default_debug -1 all [ 2545.579828] Lustre: DEBUG MARKER: oleg147-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2552.232281] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:161) [ 2558.291636] Lustre: *** cfs_fail_loc=159, val=0*** [ 2558.294286] Lustre: Skipped 1 previous similar message [ 2573.862396] Lustre: lustre-MDT0000: Client 874381a7-a049-47cb-93c0-4db0b6cd22cc (at 192.168.201.47@tcp) reconnecting [ 2593.250832] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2593.251398] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2593.253384] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2593.256085] LustreError: Skipped 1 previous similar message [ 2593.260742] Lustre: Skipped 1 previous similar message [ 2596.127132] Lustre: server umount lustre-OST0000 complete [ 2598.368861] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2598.370568] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2598.376785] Lustre: Skipped 1 previous similar message [ 2598.382402] Lustre: Skipped 2 previous similar messages [ 2603.488681] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2603.492393] Lustre: Skipped 1 previous similar message [ 2608.609098] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2608.612730] Lustre: Skipped 1 previous similar message [ 2612.191100] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2612.257300] Lustre: server umount lustre-MDT0000 complete [ 2612.705587] LustreError: 139265:0:(ldlm_lib.c:1178:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2612.713243] LustreError: 139265:0:(ldlm_lib.c:1178:target_handle_connect()) Skipped 3 previous similar messages [ 2613.817666] LustreError: 139247:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1773104798 with bad export cookie 7166643276113291970 [ 2613.820985] LustreError: MGC192.168.201.147@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2613.823101] LustreError: 139247:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2613.955633] Lustre: server umount lustre-MDT0001 complete [ 2618.525891] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing unload_modules_local [ 2619.788086] Key type lgssc unregistered [ 2619.932802] LNet: 142598:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2619.936451] LNetError: 142598:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2619.951394] LNet: Removed LNI 192.168.201.147@tcp [ 2620.313245] Key type .llcrypt unregistered [ 2620.314737] Key type ._llcrypt unregistered [ 2631.214128] Key type ._llcrypt registered [ 2631.215425] Key type .llcrypt registered [ 2631.264342] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing load_modules_local [ 2631.667083] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2631.703330] alg: No test for adler32 (adler32-zlib) [ 2632.597542] Lustre: Lustre: Build Version: 2.17.50_236_gea5aa0c [ 2632.693356] LNet: Added LNI 192.168.201.147@tcp [8/256/0/180] [ 2634.287159] Key type lgssc registered [ 2634.760716] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2637.966784] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 2637.978883] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2639.196265] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2640.929755] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing set_default_debug -1 all [ 2644.673991] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2644.881842] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2646.655172] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing set_default_debug -1 all [ 2649.174736] Lustre: DEBUG MARKER: oleg147-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2650.754087] Lustre: DEBUG MARKER: oleg147-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2653.797861] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2653.973315] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2655.016634] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:161) [ 2656.461771] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing set_default_debug -1 all [ 2658.710862] Lustre: DEBUG MARKER: oleg147-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2664.232749] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:170 to 0x280000401:193) [ 2682.591445] Lustre: lustre-OST0000: evicting c48504a8-d79b-426f-ac5a-93b348efbffc (at 192.168.201.47@tcp) by administrative request [ 2682.596874] Lustre: Skipped 1 previous similar message [ 2710.497450] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2710.499828] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2710.502559] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2710.507204] Lustre: Skipped 1 previous similar message [ 2710.512153] Lustre: Skipped 1 previous similar message [ 2713.240781] Lustre: server umount lustre-OST0000 complete [ 2715.617352] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2715.618052] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2715.624967] Lustre: Skipped 1 previous similar message [ 2720.736514] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2720.739485] Lustre: Skipped 2 previous similar messages [ 2720.871430] Lustre: server umount lustre-MDT0000 complete [ 2722.342438] LustreError: 144006:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1773104906 with bad export cookie 7461324263214522703 [ 2722.348094] LustreError: MGC192.168.201.147@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2722.348698] LustreError: 144006:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2722.483327] Lustre: server umount lustre-MDT0001 complete [ 2726.868521] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing unload_modules_local [ 2728.066980] Key type lgssc unregistered [ 2728.211294] LNet: 147719:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2728.214270] LNetError: 147719:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2728.222338] LNet: Removed LNI 192.168.201.147@tcp [ 2728.576432] Key type .llcrypt unregistered [ 2728.579356] Key type ._llcrypt unregistered [ 2758.077944] Key type ._llcrypt registered [ 2758.079427] Key type .llcrypt registered [ 2758.122204] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing load_modules_local [ 2758.548478] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2758.583094] alg: No test for adler32 (adler32-zlib) [ 2759.487304] Lustre: Lustre: Build Version: 2.17.50_236_gea5aa0c [ 2759.592857] LNet: Added LNI 192.168.201.147@tcp [8/256/0/180] [ 2761.191144] Key type lgssc registered [ 2761.625692] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2764.845342] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 2764.854833] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2766.103078] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2767.763447] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing set_default_debug -1 all [ 2771.543593] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2771.712698] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2773.318638] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing set_default_debug -1 all [ 2775.501880] Lustre: DEBUG MARKER: oleg147-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2777.023053] Lustre: DEBUG MARKER: oleg147-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2779.759966] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2779.923147] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2780.968097] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:193) [ 2782.301393] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing set_default_debug -1 all [ 2784.538815] Lustre: DEBUG MARKER: oleg147-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2791.209638] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:195 to 0x280000401:225) [ 2801.637364] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2801.638841] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2801.640551] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2801.641323] LustreError: Skipped 1 previous similar message [ 2801.654463] Lustre: Skipped 1 previous similar message [ 2805.347160] Lustre: server umount lustre-OST0000 complete [ 2807.776434] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2807.781909] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2807.788517] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2807.792183] Lustre: Skipped 1 previous similar message [ 2811.872210] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2811.872980] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2813.223982] Lustre: server umount lustre-MDT0000 complete [ 2814.882860] LustreError: 150143:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1773104999 with bad export cookie 7951754986023071039 [ 2814.886847] LustreError: MGC192.168.201.147@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2814.889405] LustreError: 150143:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2815.055563] Lustre: server umount lustre-MDT0001 complete [ 2819.782989] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing unload_modules_local [ 2821.163977] Key type lgssc unregistered [ 2821.333864] LNet: 153373:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2821.338070] LNetError: 153373:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2821.344578] LNet: Removed LNI 192.168.201.147@tcp [ 2821.736469] Key type .llcrypt unregistered [ 2821.738292] Key type ._llcrypt unregistered [ 2833.177870] Key type ._llcrypt registered [ 2833.180066] Key type .llcrypt registered [ 2833.229959] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing load_modules_local [ 2833.689074] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2833.754060] alg: No test for adler32 (adler32-zlib) [ 2834.644210] Lustre: Lustre: Build Version: 2.17.50_236_gea5aa0c [ 2834.767917] LNet: Added LNI 192.168.201.147@tcp [8/256/0/180] [ 2836.367236] Key type lgssc registered [ 2836.931871] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2844.161401] Lustre: DEBUG MARKER: == conf-sanity test 100: check lshowmount lists MGS, MDT, OST and 0@lo ========================================================== 21:10:27 (1773105027) [ 2851.400558] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing set_hostid [ 2855.793458] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing load_modules_local [ 2871.837125] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 2871.849072] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2872.980210] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2873.006598] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2873.059435] Lustre: lustre-MDT0000: new disk, initializing [ 2873.097298] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2873.110491] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2874.850550] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing set_default_debug -1 all [ 2880.066599] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2880.112754] Lustre: 157559:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2880.135673] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 2880.137681] Lustre: Skipped 1 previous similar message [ 2880.182475] Lustre: lustre-MDT0001: new disk, initializing [ 2880.227425] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2880.246986] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 2880.252842] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2881.938686] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing set_default_debug -1 all [ 2885.727864] Lustre: DEBUG MARKER: oleg147-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2887.350087] Lustre: DEBUG MARKER: oleg147-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2890.165823] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2890.297419] Lustre: lustre-OST0000: new disk, initializing [ 2890.300119] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 2890.344796] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2892.659458] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing set_default_debug -1 all [ 2896.189965] Lustre: DEBUG MARKER: oleg147-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2896.895685] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 2896.902450] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2896.948163] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2907.106185] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2907.106347] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2907.107544] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2907.111065] Lustre: Skipped 1 previous similar message [ 2911.458148] Lustre: server umount lustre-OST0000 complete [ 2916.320253] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2916.324307] LustreError: Skipped 1 previous similar message [ 2916.326540] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2916.332930] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2916.335788] Lustre: Skipped 1 previous similar message [ 2917.344355] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2917.345079] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2919.156788] Lustre: server umount lustre-MDT0000 complete [ 2920.871314] LustreError: 157550:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1773105105 with bad export cookie 5154438438085724187 [ 2920.875457] LustreError: MGC192.168.201.147@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2920.877537] LustreError: 157550:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2921.023688] Lustre: server umount lustre-MDT0001 complete [ 2925.570606] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing unload_modules_local [ 2926.874893] Key type lgssc unregistered [ 2927.024420] LNet: 161075:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2927.027329] LNetError: 161075:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2927.037338] LNet: Removed LNI 192.168.201.147@tcp [ 2927.412787] Key type .llcrypt unregistered [ 2927.414653] Key type ._llcrypt unregistered [ 2938.184932] Key type ._llcrypt registered [ 2938.187276] Key type .llcrypt registered [ 2938.244319] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing load_modules_local [ 2938.787421] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2938.800585] alg: No test for adler32 (adler32-zlib) [ 2939.681632] Lustre: Lustre: Build Version: 2.17.50_236_gea5aa0c [ 2939.781355] LNet: Added LNI 192.168.201.147@tcp [8/256/0/180] [ 2941.391122] Key type lgssc registered [ 2941.855656] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2945.269617] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 2945.277301] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2946.503573] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2948.055215] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing set_default_debug -1 all [ 2951.354820] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2951.497780] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2953.052587] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing set_default_debug -1 all [ 2955.313358] Lustre: DEBUG MARKER: oleg147-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2956.778348] Lustre: DEBUG MARKER: oleg147-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2959.458705] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2959.592245] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2961.731787] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing set_default_debug -1 all [ 2963.757362] Lustre: DEBUG MARKER: oleg147-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2970.856937] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:3 to 0x280000401:33) [ 2976.795328] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 2976.804578] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2976.812770] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 2976.816476] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2976.821252] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 2977.210636] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2977.594248] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 2977.598523] Lustre: Skipped 1 previous similar message [ 2977.612219] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2977.616171] Lustre: Skipped 1 previous similar message [ 2977.618303] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 2977.621304] Lustre: Skipped 1 previous similar message [ 2977.623679] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2977.628407] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 2977.631181] Lustre: Skipped 1 previous similar message [ 2978.013631] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2978.429639] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2978.849296] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 2978.853045] Lustre: Skipped 2 previous similar messages [ 2978.863413] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2978.870974] Lustre: Skipped 2 previous similar messages [ 2978.876878] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 2978.879231] Lustre: Skipped 2 previous similar messages [ 2978.880915] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2978.885288] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 2978.887851] Lustre: Skipped 2 previous similar messages [ 2979.271230] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2979.658862] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2980.050824] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2980.425501] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2980.837300] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2981.244836] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 2981.247605] Lustre: Skipped 5 previous similar messages [ 2981.255596] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2981.262458] Lustre: Skipped 5 previous similar messages [ 2981.264260] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 2981.267370] Lustre: Skipped 5 previous similar messages [ 2981.269430] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2981.274380] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 2981.279223] Lustre: Skipped 5 previous similar messages [ 2981.650034] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2982.039733] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2982.408496] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2982.772179] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2983.170650] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2983.569225] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2983.934365] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2984.305607] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2984.692472] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2985.060610] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2985.413922] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 2985.417594] Lustre: Skipped 10 previous similar messages [ 2985.426275] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2985.433079] Lustre: Skipped 10 previous similar messages [ 2985.435263] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 2985.438440] Lustre: Skipped 10 previous similar messages [ 2985.440627] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2985.445099] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 2985.449437] Lustre: Skipped 10 previous similar messages [ 2985.858516] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2986.266017] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2986.661607] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2987.056408] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2987.433370] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2987.791792] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2988.158351] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2988.531037] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2988.925278] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2989.324323] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2989.762058] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2990.124228] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2990.497404] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2990.820456] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2991.161101] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2991.503474] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2991.883854] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2992.248064] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2992.598193] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2992.969066] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2993.336373] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2993.694776] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 2993.697216] Lustre: Skipped 21 previous similar messages [ 2993.703992] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2993.707677] Lustre: Skipped 21 previous similar messages [ 2993.709580] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 2993.712103] Lustre: Skipped 21 previous similar messages [ 2993.714019] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2993.718912] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 2993.721429] Lustre: Skipped 21 previous similar messages [ 2994.075619] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2994.474933] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2994.847310] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2995.174842] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2995.505915] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2995.863559] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2996.231284] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2996.600693] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2996.973037] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2997.350218] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2997.715912] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2998.079348] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2998.450492] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2998.815504] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2999.166856] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2999.541253] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2999.898283] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3000.273829] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3000.646305] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3000.998263] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3001.341965] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3001.712848] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3002.068485] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3002.395278] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3002.710081] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3003.065895] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3003.392388] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3003.735261] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3004.073667] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3004.436527] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3004.835866] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3005.197527] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3005.548738] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3005.887881] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3006.215669] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3006.588844] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3006.927257] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3007.267606] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3007.607179] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3007.923374] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3008.254131] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3008.610504] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3008.962063] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3009.295431] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3009.660547] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3010.031090] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 3010.033423] Lustre: Skipped 45 previous similar messages [ 3010.040370] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3010.048280] Lustre: Skipped 45 previous similar messages [ 3010.051130] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 3010.054294] Lustre: Skipped 45 previous similar messages [ 3010.056540] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3010.061762] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 3010.064629] Lustre: Skipped 45 previous similar messages [ 3010.392265] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3010.720646] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3011.046585] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3011.367057] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3011.706612] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3012.071603] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3012.423770] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3012.795472] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3013.146276] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3013.507343] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3013.886219] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3014.268681] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3014.632570] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3015.025071] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3015.407169] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3015.756436] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3016.086618] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3016.422792] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3016.786304] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3017.147044] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3017.519670] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3017.869245] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3018.214091] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3018.569398] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3018.920213] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3019.271562] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3019.643044] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3020.023501] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3020.361982] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3020.711706] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3021.068916] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3021.432676] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3021.796371] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3022.131252] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3022.492646] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3022.861230] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3023.218534] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3023.571628] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3023.956765] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3024.354435] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3024.731981] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3025.151601] LustreError: 162468:0:(osp_precreate.c:651:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 3025.156174] LustreError: 162468:0:(osp_precreate.c:1355:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 3025.157321] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3025.519459] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3025.850647] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3026.214351] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3026.567443] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3026.912542] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3027.256675] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3027.623039] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3027.973789] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3028.330168] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3028.698798] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3029.093043] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3029.483363] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3029.918093] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3030.333922] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3030.722925] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3031.061423] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3031.414691] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3031.771107] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3032.130230] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3032.479932] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3032.814344] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3033.188198] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3033.524475] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3033.868436] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3034.198147] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3034.519833] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3034.874204] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3035.268472] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3035.633562] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3035.961904] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3036.307765] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3036.650209] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3037.011337] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3037.342135] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3037.701819] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3038.039667] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3038.368952] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3038.738333] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3039.089032] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3039.442653] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3039.795123] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3040.188560] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3040.532889] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3040.868082] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3041.214986] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3041.583355] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3041.919980] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3042.252949] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 3042.255042] Lustre: Skipped 89 previous similar messages [ 3042.261357] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3042.264862] Lustre: Skipped 89 previous similar messages [ 3042.266559] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 3042.268692] Lustre: Skipped 89 previous similar messages [ 3042.270330] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3042.273705] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 3042.275524] Lustre: Skipped 89 previous similar messages [ 3042.603628] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3042.927890] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3043.259228] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3043.579555] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3043.918042] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3044.257709] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3044.584617] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3044.930075] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3045.298965] LustreError: 162468:0:(osp_precreate.c:651:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 3045.303435] LustreError: 162468:0:(osp_precreate.c:1355:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 3045.304944] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3045.647849] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3045.984231] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3046.342666] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3046.690968] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3047.049176] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3047.392358] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3047.750358] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3048.077452] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3048.413980] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3048.759714] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3049.127956] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3049.445779] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3049.783648] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3050.122674] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3050.490972] LustreError: 162468:0:(osp_precreate.c:651:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 3050.494632] LustreError: 162468:0:(osp_precreate.c:1355:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 3050.496390] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3050.839152] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3051.190050] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3051.540458] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3051.905479] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3052.256923] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3052.601527] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3052.936566] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3053.282925] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3053.617065] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3053.972508] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3054.319518] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3054.671372] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3055.018788] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3055.379676] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3055.731981] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3056.090298] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3056.452727] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3056.820466] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3057.172897] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3057.538747] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3057.899867] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3058.272364] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3058.644767] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3058.990150] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3059.348369] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3059.713857] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3060.051798] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3060.391889] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3060.744205] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3061.123785] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3061.525768] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3061.917580] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3062.298750] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3062.677422] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3063.034312] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3063.391648] LustreError: 162468:0:(osp_precreate.c:651:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 3063.395315] LustreError: 162468:0:(osp_precreate.c:1355:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 3063.397059] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3063.737927] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3064.089178] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3064.423324] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3064.802125] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3065.170227] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3065.542783] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3065.910175] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3066.293093] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3066.667662] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3067.033873] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3067.405341] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3067.776453] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3068.139294] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3068.502518] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3068.873310] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3069.230932] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3069.592590] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3069.963121] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3070.327751] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3070.699587] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3071.076366] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3071.432410] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3071.788619] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3072.160493] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3072.540032] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3072.900483] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3073.288665] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3073.671970] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3074.038230] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3074.422239] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3074.798889] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3075.193323] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3075.558722] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3075.922507] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3076.332335] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3076.741703] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3077.113451] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3077.507310] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3077.934487] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3078.330183] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3078.710719] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3079.071771] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3079.425883] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3079.785411] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3080.151428] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3080.498924] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3080.876715] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3081.236888] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3081.592130] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3081.973199] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3082.318762] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3082.667881] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3083.015859] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3083.373844] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3083.746613] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3084.121076] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3084.529380] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3084.923322] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3085.295399] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3085.687194] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3086.075441] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3086.456964] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3086.842606] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3087.200051] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3087.561483] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3087.928581] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3088.277667] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3088.626023] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3088.986809] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3089.353454] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3089.701087] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3090.038478] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3090.363896] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3090.724633] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3091.063288] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3091.406824] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3091.747281] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3092.110951] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3092.477124] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3092.838091] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3093.189344] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3093.533940] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3093.871778] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3094.220487] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3094.595714] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3094.928507] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3095.256348] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3095.600927] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3095.979039] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3096.329771] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3096.667615] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3097.001635] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3097.361355] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3097.727031] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3098.066142] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3098.441947] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3098.808398] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3099.167442] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3099.509309] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3099.864212] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3100.194094] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3100.532855] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3100.887682] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3101.212735] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3101.555622] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3101.880185] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3102.210507] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3102.560277] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3102.910069] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3103.257766] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3103.599925] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3103.932897] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3104.270430] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3104.605374] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3104.954442] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3105.282438] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3105.599504] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3105.922613] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3106.271889] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 3106.275482] Lustre: Skipped 178 previous similar messages [ 3106.281661] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3106.285494] Lustre: Skipped 178 previous similar messages [ 3106.286621] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 3106.288113] Lustre: Skipped 178 previous similar messages [ 3106.289711] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3106.294486] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 3106.297492] Lustre: Skipped 178 previous similar messages [ 3106.634865] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3106.972685] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3107.295945] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3107.664071] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3108.001474] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3108.336108] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3108.667288] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3109.010449] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3109.344622] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3109.662507] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3110.020031] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3110.365070] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3110.692755] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3111.007386] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3111.335396] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3111.687469] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3112.023376] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3112.352419] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3112.722740] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3113.114551] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3113.452197] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3113.786028] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3114.124526] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3114.522057] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3114.854958] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3115.199958] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3115.535156] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3115.916658] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3116.252366] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3116.584630] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3116.924859] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3117.269736] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3117.643171] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3118.043392] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3118.399139] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3118.724657] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3119.077267] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3119.405218] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3119.784512] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3120.118267] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3120.453889] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3120.800570] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3121.137055] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3121.458104] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3121.774737] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3122.090894] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3122.432464] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3122.756749] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3123.090348] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3123.432750] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3123.787240] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3124.096233] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3124.420715] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3124.741367] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3125.075991] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3125.401869] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3125.716118] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3126.065960] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3126.422858] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3126.730680] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3127.049841] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3127.358352] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3127.708372] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3128.046939] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3128.407521] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3128.750920] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3129.056764] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3129.361495] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3129.696141] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3130.060689] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3130.391031] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3130.705339] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3131.024382] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3131.380299] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3131.720052] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3132.034535] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3132.344567] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3132.690835] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3133.034453] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3133.395760] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3133.802104] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3134.152032] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3134.491422] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3134.822337] LustreError: 162468:0:(osp_precreate.c:651:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 3134.826694] LustreError: 162468:0:(osp_precreate.c:1355:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 3134.827961] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3135.180081] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3135.537041] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3135.880322] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3136.217140] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3136.566876] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3136.931994] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3137.308042] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3137.649304] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3137.987188] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3138.324906] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3138.651497] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3139.090844] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3139.485491] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3139.856896] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3140.215817] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3140.569939] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3140.929697] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3141.290294] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3141.616129] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3141.966084] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3142.342583] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3142.711816] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3143.090736] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3143.442885] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3143.779632] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3144.130945] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3144.476396] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3144.789696] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3145.113575] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3145.436410] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3145.776529] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3146.176295] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3146.531867] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3146.889347] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3147.211796] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3147.553619] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3147.903626] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3148.248496] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3148.597918] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3148.937993] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3149.279570] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3149.632204] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3150.017444] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3150.335372] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3150.669415] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3151.040931] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3151.418939] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3151.802063] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3152.154608] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3152.501540] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3152.865633] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3153.199905] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3153.525443] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3153.897115] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3154.275584] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3154.599391] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3154.930413] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3155.235586] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3155.555401] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3155.860101] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3156.178042] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3156.487025] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3156.839275] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3157.173613] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3157.514278] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3157.852435] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3158.233906] LustreError: 162468:0:(osp_precreate.c:651:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 3158.238101] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3158.238832] LustreError: 162468:0:(osp_precreate.c:1355:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 3158.557444] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3158.886072] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3159.214342] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3159.554556] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3159.861311] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3160.164955] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3160.479377] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3160.795554] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3161.087751] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3161.379333] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3161.688754] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3162.009552] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3162.335932] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3162.635319] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3162.953161] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3163.319576] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3163.648868] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3164.005642] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3164.378675] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3164.776874] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3165.127956] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3165.460612] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3165.808858] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3166.139197] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3166.477677] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3166.825643] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3167.145854] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3167.485397] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3167.799716] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3168.178487] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3168.498669] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3168.834214] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3169.186726] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3169.541050] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3169.843798] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3170.164695] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3170.475742] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3170.786360] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3171.096303] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3171.416247] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3171.709141] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3172.035780] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3172.414967] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3172.795395] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3173.151452] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3173.520567] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3173.896038] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3174.267329] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3174.596737] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3174.926581] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3175.259505] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3175.573672] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3175.872479] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3176.195656] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3176.530447] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3176.832554] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3177.150053] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3177.459136] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3177.803458] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3178.145377] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3178.501115] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3178.872442] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3179.244402] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3179.580187] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3179.905530] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3180.263678] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3180.587831] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3180.915673] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3181.239195] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3181.549391] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3181.865432] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3182.190419] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3182.533691] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3182.870899] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3183.217328] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3183.600023] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3183.953801] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3184.292363] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3184.650302] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3185.010584] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3185.323830] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3185.635515] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3185.975497] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3186.284738] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3186.601620] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3186.915038] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3187.287479] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3187.599202] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3187.920870] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3188.273365] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3188.668295] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3189.054562] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3189.448326] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3189.811479] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3190.139293] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3190.462634] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3190.777193] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3191.113086] LustreError: 162468:0:(osp_precreate.c:651:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 3191.117152] LustreError: 162468:0:(osp_precreate.c:651:osp_precreate_send()) Skipped 1 previous similar message [ 3191.118455] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3191.119069] LustreError: 162468:0:(osp_precreate.c:1355:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 3191.125869] LustreError: 162468:0:(osp_precreate.c:1355:osp_precreate_thread()) Skipped 1 previous similar message [ 3191.417179] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3191.732343] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3192.038951] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3192.366680] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3192.677358] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3192.999211] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3193.297112] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3193.654774] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3193.966492] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3194.325545] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3194.647079] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3194.981992] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3195.288926] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3195.598138] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3195.937825] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3196.262964] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3196.582209] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3196.938077] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3197.282762] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3197.639535] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3197.976477] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3198.315857] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3198.660890] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3199.006731] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3199.330938] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3199.643250] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3199.957598] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3200.294642] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3200.590184] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3200.902339] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3201.216725] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3201.559040] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3201.875865] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3202.202969] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3202.514821] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3202.863520] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3203.196780] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3203.518947] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3203.859505] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3204.212982] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3204.524116] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3204.844121] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3205.162749] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3205.519403] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3205.829059] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3206.161073] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3206.481248] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3206.803934] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3207.131179] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3207.475530] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3207.801538] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3208.183336] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3208.532910] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3208.872319] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3209.228222] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3209.548962] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3209.862461] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3210.182083] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3210.531212] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3210.852945] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3211.167761] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3211.495101] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3211.842352] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3212.182437] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3212.530587] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3212.882094] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3213.254624] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3213.578434] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3213.904705] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3214.236876] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3214.588532] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3214.927343] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3215.233955] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3215.540679] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3215.874063] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3216.174803] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3216.484127] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3216.796127] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3217.136051] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3217.458971] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3217.784066] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3218.133086] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3218.510662] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3218.857910] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3219.191059] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3219.521604] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3219.854564] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3220.155235] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3220.474809] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3220.812975] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3221.181536] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3221.523000] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3221.866028] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3222.196114] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3222.550328] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3222.879574] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3223.216435] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3223.543155] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3223.889633] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3224.211579] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3224.518462] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3224.857452] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3225.218234] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3225.537931] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3225.873431] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3226.225127] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3226.607828] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3226.947992] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3227.335310] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3227.753425] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3228.151746] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3228.565764] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3228.986289] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3229.349316] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3229.737124] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3230.094664] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3230.441761] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3230.799693] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3231.182988] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3231.555712] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3231.970059] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3232.392412] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3232.768563] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3233.138565] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3233.529081] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3233.907859] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3234.250065] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3234.642983] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 3234.646577] Lustre: Skipped 376 previous similar messages [ 3234.654918] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3234.661065] Lustre: Skipped 376 previous similar messages [ 3234.662950] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 3234.665892] Lustre: Skipped 376 previous similar messages [ 3234.668472] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3234.672429] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 3234.675443] Lustre: Skipped 376 previous similar messages [ 3235.038525] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3235.381015] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3235.746114] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3236.093829] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3236.441175] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3236.786572] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3237.111454] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3237.443023] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3237.793956] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3238.141337] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3238.510149] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3238.871972] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3239.242623] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3239.598312] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3239.951168] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3240.286073] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3240.614654] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3240.949043] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3241.304667] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3241.634704] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3241.949674] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3242.268187] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3242.579379] LustreError: 162468:0:(osp_precreate.c:651:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 3242.583106] LustreError: 162468:0:(osp_precreate.c:1355:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 3242.584204] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3242.912388] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3243.247764] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3243.568859] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3243.924328] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3244.264257] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3244.618533] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3244.963064] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3245.316715] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3245.655740] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3246.004690] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3246.376876] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3246.712647] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3247.022611] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3247.343694] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3247.649540] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3247.968314] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3248.317508] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3248.645940] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3248.980731] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3249.301822] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3249.691512] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3250.009243] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3250.329661] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3250.648082] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3250.985188] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3251.306914] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3251.621326] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3251.961744] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3252.365725] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3252.727647] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3253.083675] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3253.428473] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3253.757403] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3254.126773] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3254.488789] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3254.832346] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3255.191239] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3255.556127] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3255.926679] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3256.290623] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3256.704542] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3257.065732] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3257.416544] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3257.796305] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3258.145919] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3258.501405] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3258.836152] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3259.224222] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3259.568042] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3259.946729] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3260.313947] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3260.663352] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3261.022402] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3261.357419] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3261.682448] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3262.003295] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3262.323605] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3262.667725] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3263.012174] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3263.358283] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3263.711812] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3264.071726] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3264.419153] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3264.734052] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3265.058594] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3265.415943] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3265.738132] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3266.047037] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3266.356202] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3266.704489] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3267.039892] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3267.395793] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3267.716653] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3268.074927] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3268.381176] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3268.702673] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3269.002284] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3269.321236] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3269.616994] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3269.905432] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3270.218822] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3270.554733] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3270.878840] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3271.194906] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3271.501511] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3271.814251] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3272.129062] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3272.440282] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3272.738706] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3273.039846] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3273.358745] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3273.667481] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3273.976516] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3274.316648] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3274.621014] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3274.925083] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3275.254525] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3275.588957] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3275.902885] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3276.226791] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3276.556314] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3276.914742] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3277.261936] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3277.616413] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3277.943282] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3278.260521] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3278.568757] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3278.902037] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3279.260630] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3279.639598] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3279.981809] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3280.311512] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3280.651060] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3280.980869] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3281.315329] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3281.661545] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3281.996301] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3282.369827] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3282.718815] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3283.059238] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3283.442731] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3283.834480] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3284.198480] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3284.528608] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3284.886609] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3285.250974] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3285.623626] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3285.999941] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3286.378484] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3286.741886] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3287.135283] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3287.546210] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3287.924689] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3288.280321] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3288.634990] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3289.031280] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3289.398718] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3289.767301] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3290.122604] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3290.471627] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3290.816890] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3291.165238] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3291.552469] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3291.919231] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3292.252105] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3292.583637] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3292.935984] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3293.263429] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3293.601873] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3293.921716] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3294.268024] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3294.588354] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3294.932656] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3295.265630] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3295.608053] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3295.951152] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3296.335329] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3296.680768] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3297.020434] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3297.339695] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3297.688284] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3298.029726] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3298.363027] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3298.700371] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3299.048055] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3299.389210] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3299.732278] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3300.120702] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3300.456379] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3300.792344] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3301.131708] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3301.485434] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3301.846461] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3302.226112] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3302.583832] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3302.928503] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3303.274872] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3303.607902] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3303.940905] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3304.277490] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3304.615988] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3304.946487] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3305.291046] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3305.631427] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3305.987069] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3306.319041] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3306.660469] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3306.984566] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3307.321610] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3307.667773] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3308.014272] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3308.350248] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3308.711149] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3309.069618] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3309.418468] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3309.807469] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3310.181138] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3310.512152] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3310.834381] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3311.168434] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3311.506751] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3311.914523] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3312.264726] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3312.591954] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3312.937758] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3313.275522] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3313.628180] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3313.989278] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3314.340834] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3314.679422] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3315.035357] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3315.385567] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3315.727633] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3316.065863] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3316.417339] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3316.765346] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3317.094067] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3317.431338] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3317.773952] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3318.084364] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3318.384633] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3318.679034] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3318.989672] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3319.288542] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3319.611895] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3319.941408] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3320.285565] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3320.612564] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3320.913832] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3321.189678] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3321.510115] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3321.815270] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3322.104283] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3322.379499] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3322.673323] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3322.953655] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3323.246282] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3323.548653] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3323.872513] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3324.178488] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3417.014752] LustreError: 164605:0:(ofd_dev.c:1780:ofd_create_hdl()) lustre-OST0000: unable to precreate: rc = -28 [ 3417.019129] LustreError: 162468:0:(osp_precreate.c:651:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -28 [ 3417.023210] LustreError: 162468:0:(osp_precreate.c:651:osp_precreate_send()) Skipped 2 previous similar messages [ 3518.432672] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3518.437760] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3518.446984] Lustre: Skipped 263 previous similar messages [ 3518.450133] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3522.144698] Lustre: server umount lustre-OST0000 complete [ 3524.069517] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3524.078424] Lustre: Skipped 2 previous similar messages [ 3529.184297] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3529.186929] Lustre: Skipped 1 previous similar message [ 3534.305139] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3534.308118] Lustre: Skipped 1 previous similar message [ 3538.399132] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 3538.551528] Lustre: server umount lustre-MDT0000 complete [ 3539.425842] LustreError: 162443:0:(ldlm_lib.c:1178:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3539.434754] LustreError: 162443:0:(ldlm_lib.c:1178:target_handle_connect()) Skipped 3 previous similar messages [ 3540.279737] LustreError: 162431:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1773105724 with bad export cookie 11978605234882043300 [ 3540.283025] LustreError: MGC192.168.201.147@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3540.284829] LustreError: 162431:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 3540.425338] Lustre: server umount lustre-MDT0001 complete [ 3545.019673] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing unload_modules_local [ 3546.358203] Key type lgssc unregistered [ 3546.525325] LNet: 217135:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3546.527938] LNetError: 217135:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3546.538318] LNet: Removed LNI 192.168.201.147@tcp [ 3546.903150] Key type .llcrypt unregistered [ 3546.905124] Key type ._llcrypt unregistered [ 3558.853387] Key type ._llcrypt registered [ 3558.855193] Key type .llcrypt registered [ 3558.913400] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing load_modules_local [ 3559.485256] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3559.532701] alg: No test for adler32 (adler32-zlib) [ 3560.442511] Lustre: Lustre: Build Version: 2.17.50_236_gea5aa0c [ 3560.576533] LNet: Added LNI 192.168.201.147@tcp [8/256/0/180] [ 3562.183305] Key type lgssc registered [ 3562.728988] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3566.865190] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3566.884993] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3568.191678] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3570.356412] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing set_default_debug -1 all [ 3574.978521] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3575.203811] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3577.155465] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing set_default_debug -1 all [ 3579.739897] Lustre: DEBUG MARKER: oleg147-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 3581.581707] Lustre: DEBUG MARKER: oleg147-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 3584.755431] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3584.991463] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3587.643259] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing set_default_debug -1 all [ 3590.603584] Lustre: DEBUG MARKER: oleg147-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3593.197257] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49666 to 0x280000401:49697) [ 3598.299306] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 3599.174787] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3599.181958] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 3599.186894] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3599.193676] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 3599.193675] LustreError: 218530:0:(osp_precreate.c:1263:osp_precreate_thread()) cfs_fail_timeout id 2107 sleeping for 20000ms [ 3601.378539] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3601.378927] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3601.384189] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3601.389844] Lustre: Skipped 1 previous similar message [ 3606.413538] Lustre: server umount lustre-OST0000 complete [ 3619.263126] LustreError: 218530:0:(osp_precreate.c:1263:osp_precreate_thread()) cfs_fail_timeout id 2107 awake [ 3635.481640] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3635.652574] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3638.199671] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing set_default_debug -1 all [ 3640.501201] Lustre: DEBUG MARKER: oleg147-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3640.805260] LustreError: lustre-OST0000-osc-MDT0001: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3640.811221] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3640.811232] Lustre: lustre-OST0000-osc-MDT0001: Connection restored to 0@lo (at 0@lo) [ 3640.819682] Lustre: Skipped 1 previous similar message [ 3643.427636] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid in FULL state after 2 sec [ 3645.183626] Lustre: DEBUG MARKER: oleg147-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff9e7bd1082000.ost_server_uuid 50 [ 3645.863029] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff9e7bd1082000.ost_server_uuid in FULL state after 0 sec [ 3651.041105] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3651.041547] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3651.049073] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3651.056811] Lustre: Skipped 3 previous similar messages [ 3653.282713] Lustre: server umount lustre-OST0000 complete [ 3656.160508] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3656.162504] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3656.166275] Lustre: Skipped 2 previous similar messages [ 3656.172096] Lustre: Skipped 1 previous similar message [ 3661.031755] Lustre: server umount lustre-MDT0000 complete [ 3661.280468] LustreError: 218511:0:(ldlm_lib.c:1178:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3661.285264] LustreError: 218511:0:(ldlm_lib.c:1178:target_handle_connect()) Skipped 3 previous similar messages [ 3662.632516] LustreError: 218490:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1773105846 with bad export cookie 10685514145037526359 [ 3662.636531] LustreError: MGC192.168.201.147@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3662.638151] LustreError: 218490:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 3662.770283] Lustre: server umount lustre-MDT0001 complete [ 3667.409919] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing unload_modules_local [ 3668.683422] Key type lgssc unregistered [ 3668.835510] LNet: 223168:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3668.839109] LNetError: 223168:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3668.850143] LNet: Removed LNI 192.168.201.147@tcp [ 3669.271608] Key type .llcrypt unregistered [ 3669.272675] Key type ._llcrypt unregistered [ 3683.541582] Key type ._llcrypt registered [ 3683.543037] Key type .llcrypt registered [ 3683.602580] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing load_modules_local [ 3684.120456] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3684.140471] alg: No test for adler32 (adler32-zlib) [ 3685.044216] Lustre: Lustre: Build Version: 2.17.50_236_gea5aa0c [ 3685.153912] LNet: Added LNI 192.168.201.147@tcp [8/256/0/180] [ 3686.767192] Key type lgssc registered [ 3687.376642] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3693.052023] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3693.070497] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3694.542907] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3697.981242] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing set_default_debug -1 all [ 3703.787878] LustreError: 224857:0:(ldlm_lib.c:1178:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3703.839609] LustreError: 224857:0:(ldlm_lib.c:1178:target_handle_connect()) Skipped 2 previous similar messages [ 3706.800298] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3707.391814] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3712.232828] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing set_default_debug -1 all [ 3714.853589] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 3722.039032] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3722.402498] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3729.147393] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing set_default_debug -1 all [ 3738.673726] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3738.876659] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49699 to 0x280000401:49729) [ 3738.948692] Lustre: lustre-OST0001: new disk, initializing [ 3738.960756] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 3739.204191] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 3744.312951] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing set_default_debug -1 all [ 3746.844495] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 3746.857271] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 3746.897244] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 3751.947271] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 3755.725918] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 3772.386959] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3772.389546] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3772.395308] Lustre: Skipped 2 previous similar messages [ 3772.400584] Lustre: Skipped 3 previous similar messages [ 3775.641151] Lustre: server umount lustre-MDT0000 complete [ 3777.508155] LustreError: 224857:0:(ldlm_lib.c:1178:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3777.520701] LustreError: 224857:0:(ldlm_lib.c:1178:target_handle_connect()) Skipped 3 previous similar messages [ 3777.814771] LustreError: 224838:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1773105962 with bad export cookie 3720063570459712515 [ 3777.815368] LustreError: MGC192.168.201.147@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3777.822430] LustreError: 224838:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 3778.133206] Lustre: server umount lustre-MDT0001 complete [ 3790.874772] Lustre: server umount lustre-OST0000 complete [ 3803.582835] Lustre: server umount lustre-OST0001 complete [ 3817.088619] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing load_modules_local [ 3823.750569] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3824.058830] LustreError: 230194:0:(ldlm_lib.c:1178:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3824.144451] Lustre: mylustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3826.603203] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing set_default_debug -1 all [ 3829.222050] LustreError: 230195:0:(ldlm_lib.c:1178:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3832.152743] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3832.445229] Lustre: mylustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3834.786511] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing set_default_debug -1 all [ 3836.548550] Lustre: 231301:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 3840.314392] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3842.663912] LustreError: 231654:0:(ldlm_lib.c:1178:target_handle_connect()) mylustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3844.229799] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing set_default_debug -1 all [ 3849.145310] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3849.200801] Lustre: mylustre-OST0000: new connection from mylustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49731 to 0x280000401:49761) [ 3849.336635] Lustre: mylustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 3849.345019] Lustre: Skipped 1 previous similar message [ 3851.023530] Lustre: mylustre-OST0001: new connection from mylustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:33) [ 3852.601769] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing set_default_debug -1 all [ 3857.242712] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 3858.989363] Lustre: 233142:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 3873.249356] LustreError: mylustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3873.259693] Lustre: mylustre-MDT0000-osp-MDT0001: Connection to mylustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3873.272502] Lustre: Skipped 1 previous similar message [ 3873.277267] Lustre: mylustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3875.305350] Lustre: mylustre-MDT0000-lwp-OST0001: Connection to mylustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3875.309671] Lustre: mylustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3875.329134] Lustre: Skipped 2 previous similar messages [ 3875.333912] Lustre: Skipped 3 previous similar messages [ 3880.426119] Lustre: mylustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3880.435463] Lustre: Skipped 3 previous similar messages [ 3885.542292] Lustre: mylustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3885.545795] Lustre: Skipped 3 previous similar messages [ 3886.047156] Lustre: mylustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 3886.190933] Lustre: server umount mylustre-MDT0000 complete [ 3887.991595] LustreError: 230170:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1773106072 with bad export cookie 3720063570459715322 [ 3887.996603] LustreError: MGC192.168.201.147@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3887.997786] LustreError: 230170:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 3888.173795] Lustre: server umount mylustre-MDT0001 complete [ 3899.877693] Lustre: server umount mylustre-OST0000 complete [ 3912.590642] Lustre: server umount mylustre-OST0001 complete [ 3925.624148] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing load_modules_local [ 3931.685653] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3932.060264] LustreError: 235435:0:(ldlm_lib.c:1178:target_handle_connect()) tfs-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3932.068784] LustreError: 235435:0:(ldlm_lib.c:1178:target_handle_connect()) Skipped 3 previous similar messages [ 3932.167416] Lustre: tfs-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3934.946253] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing set_default_debug -1 all [ 3940.800881] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3943.832361] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing set_default_debug -1 all [ 3945.763600] Lustre: 236543:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 3950.187897] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3951.530959] LustreError: 236898:0:(ldlm_lib.c:1178:target_handle_connect()) tfs-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3951.541926] LustreError: 236898:0:(ldlm_lib.c:1178:target_handle_connect()) Skipped 1 previous similar message [ 3954.412058] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing set_default_debug -1 all [ 3957.750025] Lustre: tfs-OST0000: new connection from tfs-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49731 to 0x280000401:49793) [ 3960.143756] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3961.831229] Lustre: tfs-OST0001: new connection from tfs-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:35 to 0x2c0000401:65) [ 3964.779360] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing set_default_debug -1 all [ 3970.322042] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 3972.726297] Lustre: 238388:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 3982.304749] LustreError: tfs-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3982.311343] Lustre: tfs-MDT0000-osp-MDT0001: Connection to tfs-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3982.333811] Lustre: tfs-MDT0000: Not available for connect from 0@lo (stopping) [ 3987.934706] Lustre: server umount tfs-MDT0000 complete [ 3990.442866] LustreError: 235410:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1773106174 with bad export cookie 3720063570459717212 [ 3990.444320] LustreError: MGC192.168.201.147@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3990.447875] LustreError: 235410:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 3990.710886] Lustre: server umount tfs-MDT0001 complete [ 4003.561078] Lustre: server umount tfs-OST0000 complete [ 4016.554735] Lustre: server umount tfs-OST0001 complete [ 4030.516454] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing load_modules_local [ 4036.866465] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4037.173342] LustreError: 240619:0:(ldlm_lib.c:1178:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4037.187300] LustreError: 240619:0:(ldlm_lib.c:1178:target_handle_connect()) Skipped 3 previous similar messages [ 4037.224692] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4037.228473] Lustre: Skipped 3 previous similar messages [ 4039.411830] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing set_default_debug -1 all [ 4044.356883] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4046.927834] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing set_default_debug -1 all [ 4048.599808] Lustre: 241729:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 4052.138493] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4055.448591] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing set_default_debug -1 all [ 4060.712797] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4062.000453] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:35 to 0x2c0000401:97) [ 4062.002659] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49731 to 0x280000401:49825) [ 4064.747508] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing set_default_debug -1 all [ 4069.825521] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 4071.922407] Lustre: 243572:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 4081.082658] Lustre: DEBUG MARKER: == conf-sanity test 104a: Make sure user defined options are reflected in mount ========================================================== 21:31:04 (1773106264) [ 4095.969598] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4095.976744] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4095.983303] Lustre: Skipped 3 previous similar messages [ 4095.988871] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4095.991577] Lustre: Skipped 4 previous similar messages [ 4100.473929] Lustre: server umount lustre-MDT0000 complete [ 4102.205657] LustreError: 240595:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1773106286 with bad export cookie 3720063570459718878 [ 4102.211476] LustreError: MGC192.168.201.147@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4102.212911] LustreError: 240595:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4103.137212] LustreError: 240615:0:(ldlm_lib.c:1178:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4103.144803] LustreError: 240615:0:(ldlm_lib.c:1178:target_handle_connect()) Skipped 4 previous similar messages [ 4108.474673] Lustre: server umount lustre-MDT0001 complete [ 4116.583546] Lustre: server umount lustre-OST0000 complete [ 4133.344730] Lustre: lustre-OST0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 4133.458368] Lustre: server umount lustre-OST0001 complete [ 4137.283706] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing set_hostid [ 4142.930945] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing load_modules_local [ 4163.586385] LDISKFS-fs (dm-0): Mount option "noacl" will be removed by 3.5 [ 4163.586385] Contact linux-ldiskfs@vger.kernel.org if you think we should keep it. [ 4163.586385] [ 4163.606404] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: acl,user_xattr,errors=remount-ro,noacl,no_mbcache,nodelalloc [ 4163.810819] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 4163.843789] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 4163.908383] Lustre: lustre-MDT0000: new disk, initializing [ 4163.987865] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 4166.055886] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing set_default_debug -1 all [ 4172.194313] LDISKFS-fs (dm-1): Mount option "noacl" will be removed by 3.5 [ 4172.194313] Contact linux-ldiskfs@vger.kernel.org if you think we should keep it. [ 4172.194313] [ 4172.211644] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: acl,user_xattr,errors=remount-ro,noacl,no_mbcache,nodelalloc [ 4172.288142] Lustre: 246716:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 4172.326514] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 4172.330378] Lustre: Skipped 1 previous similar message [ 4172.382430] Lustre: lustre-MDT0001: new disk, initializing [ 4172.445096] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4172.452834] Lustre: Skipped 4 previous similar messages [ 4172.473657] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 4172.482702] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 4174.527636] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing set_default_debug -1 all [ 4180.398171] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4180.524262] Lustre: lustre-OST0000: new disk, initializing [ 4180.527238] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 4182.539936] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 4182.544861] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 4182.593012] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 4183.702992] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing set_default_debug -1 all [ 4189.651537] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4189.751398] Lustre: lustre-OST0001: new disk, initializing [ 4189.754836] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 4190.999782] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 4191.008295] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 4191.054984] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 4193.012017] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing set_default_debug -1 all [ 4199.740662] Lustre: DEBUG MARKER: == conf-sanity test 104b: Mount uses last flock argument ========================================================== 21:33:03 (1773106383) [ 4207.515545] Lustre: DEBUG MARKER: == conf-sanity test 105: check file creation for ro and rw bind mnt pt ========================================================== 21:33:11 (1773106391) [ 4221.930763] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4221.934159] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4221.938470] Lustre: Skipped 6 previous similar messages [ 4221.942724] Lustre: Skipped 8 previous similar messages [ 4226.210318] Lustre: server umount lustre-OST0000 complete [ 4229.089637] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4232.169062] LustreError: 248487:0:(ldlm_lib.c:1178:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4232.186943] LustreError: 248487:0:(ldlm_lib.c:1178:target_handle_connect()) Skipped 4 previous similar messages [ 4234.871925] Lustre: server umount lustre-MDT0000 complete [ 4237.444303] LustreError: 246708:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1773106421 with bad export cookie 3720063570459720425 [ 4237.448773] LustreError: MGC192.168.201.147@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4237.462932] LustreError: 246708:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4242.403949] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4242.410070] Lustre: Skipped 3 previous similar messages [ 4243.739919] Lustre: server umount lustre-MDT0001 complete [ 4249.883324] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing unload_modules_local [ 4263.053156] Lustre: server umount lustre-OST0001 complete [ 4266.941183] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing set_hostid [ 4276.292882] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing load_modules_local [ 4299.220060] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4299.432456] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 4299.460429] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 4299.527172] Lustre: lustre-MDT0000: new disk, initializing [ 4299.601430] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 4302.081316] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing set_default_debug -1 all [ 4309.180153] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4309.283830] Lustre: 253582:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 4309.497712] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 4312.054973] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing set_default_debug -1 all [ 4316.714245] Lustre: DEBUG MARKER: oleg147-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 4318.614468] Lustre: DEBUG MARKER: oleg147-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 4322.824650] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4323.036817] Lustre: lustre-OST0000: new disk, initializing [ 4323.042036] Lustre: Skipped 1 previous similar message [ 4323.046454] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 4323.050088] Lustre: Skipped 2 previous similar messages [ 4324.427495] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 4324.432378] Lustre: Skipped 1 previous similar message [ 4324.434575] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 4324.487939] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 4326.324869] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing set_default_debug -1 all [ 4331.214605] Lustre: DEBUG MARKER: oleg147-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4335.073433] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4335.076148] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4335.079934] LustreError: Skipped 1 previous similar message [ 4335.089446] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4335.099333] Lustre: Skipped 7 previous similar messages [ 4340.582529] Lustre: server umount lustre-OST0000 complete [ 4345.313978] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4349.005967] Lustre: server umount lustre-MDT0000 complete [ 4351.247410] LustreError: 253574:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1773106535 with bad export cookie 3720063570459721895 [ 4351.251748] LustreError: MGC192.168.201.147@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4351.256156] LustreError: 253574:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 4351.465216] Lustre: server umount lustre-MDT0001 complete [ 4357.691475] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing unload_modules_local [ 4359.408303] Key type lgssc unregistered [ 4359.596145] LNet: 256950:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4359.603713] LNetError: 256950:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4359.613488] LNet: Removed LNI 192.168.201.147@tcp [ 4360.086040] Key type .llcrypt unregistered [ 4360.087132] Key type ._llcrypt unregistered [ 4374.788369] Key type ._llcrypt registered [ 4374.790875] Key type .llcrypt registered [ 4374.847291] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing load_modules_local [ 4375.454853] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4375.471206] alg: No test for adler32 (adler32-zlib) [ 4376.398841] Lustre: Lustre: Build Version: 2.17.50_236_gea5aa0c [ 4376.526069] LNet: Added LNI 192.168.201.147@tcp [8/256/0/180] [ 4378.135534] Key type lgssc registered [ 4378.816944] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4383.262345] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4383.277937] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4384.611466] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4386.890270] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing set_default_debug -1 all [ 4391.843492] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4392.101504] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4394.526422] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing set_default_debug -1 all [ 4397.457552] Lustre: DEBUG MARKER: oleg147-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 4399.651489] Lustre: DEBUG MARKER: oleg147-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 4403.189223] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4403.383154] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4406.247717] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing set_default_debug -1 all [ 4409.267495] Lustre: DEBUG MARKER: oleg147-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4409.517693] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:3 to 0x280000401:33) [ 4409.855787] Lustre: Setting parameter lustre-OST0000.ost.unknown_param=50 in log lustre-OST0000 [ 4414.947230] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4414.950953] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4414.958580] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4416.994245] LustreError: 259974:0:(ldlm_lib.c:1178:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4417.004018] LustreError: 259974:0:(ldlm_lib.c:1178:target_handle_connect()) Skipped 2 previous similar messages [ 4417.041496] Lustre: server umount lustre-OST0000 complete [ 4422.117927] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4422.120473] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4422.130957] Lustre: Skipped 1 previous similar message [ 4427.234733] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4427.238613] Lustre: Skipped 2 previous similar messages [ 4433.379967] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4433.380332] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 4. Is it stuck? [ 4433.383320] Lustre: Skipped 3 previous similar messages [ 4433.473430] Lustre: server umount lustre-MDT0000 complete [ 4435.437858] LustreError: 258361:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1773106619 with bad export cookie 7354830931751355653 [ 4435.439812] LustreError: MGC192.168.201.147@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4435.442373] LustreError: 258361:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4435.605722] Lustre: server umount lustre-MDT0001 complete [ 4441.188053] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing unload_modules_local [ 4442.794713] Key type lgssc unregistered [ 4442.964741] LNet: 261609:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4442.968542] LNetError: 261609:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4442.978384] LNet: Removed LNI 192.168.201.147@tcp [ 4443.392220] Key type .llcrypt unregistered [ 4443.394205] Key type ._llcrypt unregistered [ 4452.761917] Key type ._llcrypt registered [ 4452.763814] Key type .llcrypt registered [ 4452.817977] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing load_modules_local [ 4453.323909] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4453.388833] alg: No test for adler32 (adler32-zlib) [ 4454.275863] Lustre: Lustre: Build Version: 2.17.50_236_gea5aa0c [ 4454.406083] LNet: Added LNI 192.168.201.147@tcp [8/256/0/180] [ 4456.023162] Key type lgssc registered [ 4456.582962] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4461.332613] Lustre: lustre-OST0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4461.346796] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4476.831363] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 4481.951663] LustreError: 262766:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.201.147@tcp: failed processing log, type 4: rc = -110 [ 4508.575263] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4511.375249] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing set_default_debug -1 all [ 4513.852664] Lustre: DEBUG MARKER: oleg147-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4514.890212] Lustre: server umount lustre-OST0000 complete [ 4522.544364] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing unload_modules_local [ 4523.908151] Key type lgssc unregistered [ 4524.066093] LNet: 264264:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4524.068562] LNetError: 264264:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4524.080339] LNet: Removed LNI 192.168.201.147@tcp [ 4524.437402] Key type .llcrypt unregistered [ 4524.438923] Key type ._llcrypt unregistered [ 4543.010462] Key type ._llcrypt registered [ 4543.012816] Key type .llcrypt registered [ 4543.065337] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing load_modules_local [ 4543.508923] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4543.545169] alg: No test for adler32 (adler32-zlib) [ 4544.453818] Lustre: Lustre: Build Version: 2.17.50_236_gea5aa0c [ 4544.567188] LNet: Added LNI 192.168.201.147@tcp [8/256/0/180] [ 4546.168121] Key type lgssc registered [ 4546.792220] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4552.712802] loop0: detected capacity change from 0 to 204800000 [ 4552.784350] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 4552.788702] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 4552.942171] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 4553.410445] loop0: detected capacity change from 0 to 204800000 [ 4553.483082] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 4553.490428] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 4553.676596] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 4554.118147] loop0: detected capacity change from 0 to 204800000 [ 4554.233130] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 4554.239963] blk_update_request: operation not supported error, dev loop0, sector 8224 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 4554.294436] blk_update_request: operation not supported error, dev loop0, sector 58264 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 4554.792912] loop0: detected capacity change from 0 to 204800000 [ 4554.874518] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 4555.446159] loop0: detected capacity change from 0 to 204800000 [ 4555.456587] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 4557.312749] loop0: detected capacity change from 0 to 204800000 [ 4557.329438] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 4559.173873] loop0: detected capacity change from 0 to 204800000 [ 4559.187821] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 4561.024582] loop0: detected capacity change from 0 to 204800000 [ 4561.045874] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 4563.040410] loop0: detected capacity change from 0 to 204800000 [ 4563.077311] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4563.089606] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4563.141585] Lustre: 267637:0:(osd_scrub.c:1831:osd_index_restore()) lustre-MDT0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 4563.149271] Lustre: 267637:0:(osd_scrub.c:1831:osd_index_restore()) lustre-MDT0000: restore index '0x2010000-MDT0000' with [0x200000005:0x3:0x0]: rc = 0 [ 4563.157835] Lustre: 267637:0:(osd_scrub.c:1831:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000003:0xc:0x0]: rc = 0 [ 4563.165064] Lustre: 267637:0:(osd_scrub.c:1831:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000003:0xb:0x0]: rc = 0 [ 4563.173200] Lustre: 267637:0:(osd_scrub.c:1831:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000003:0xa:0x0]: rc = 0 [ 4563.179591] Lustre: 267637:0:(osd_scrub.c:1831:osd_index_restore()) lustre-MDT0000: restore index '0x1010000-MDT0000' with [0x200000005:0x2:0x0]: rc = 0 [ 4563.187715] Lustre: 267637:0:(osd_scrub.c:1831:osd_index_restore()) lustre-MDT0000: restore index '0x10000-MDT0000' with [0x200000005:0x1:0x0]: rc = 0 [ 4563.195045] Lustre: 267637:0:(osd_scrub.c:1831:osd_index_restore()) lustre-MDT0000: restore index 'nodemap' with [0x200000003:0x3:0x0]: rc = 0 [ 4563.202547] Lustre: 267637:0:(osd_scrub.c:1831:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000006:0x2010000:0x0]: rc = 0 [ 4563.212893] Lustre: 267637:0:(osd_scrub.c:1831:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000006:0x1010000:0x0]: rc = 0 [ 4563.221494] Lustre: 267637:0:(osd_scrub.c:1831:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000006:0x10000:0x0]: rc = 0 [ 4563.228181] Lustre: 267637:0:(osd_scrub.c:1831:osd_index_restore()) lustre-MDT0000: restore index '0x20000' with [0x200000006:0x20000:0x0]: rc = 0 [ 4563.235572] Lustre: 267637:0:(osd_scrub.c:1831:osd_index_restore()) lustre-MDT0000: restore index '0x1020000' with [0x200000006:0x1020000:0x0]: rc = 0 [ 4563.243448] Lustre: 267637:0:(osd_scrub.c:1831:osd_index_restore()) lustre-MDT0000: restore index '0x2020000' with [0x200000006:0x2020000:0x0]: rc = 0 [ 4566.441244] Lustre: server umount MGS complete [ 4566.834256] loop0: detected capacity change from 0 to 204800000 [ 4566.868590] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4567.103532] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4567.131139] Lustre: lustre-MDT0000: Aborting client recovery [ 4567.549894] loop1: detected capacity change from 0 to 204800000 [ 4567.598411] LDISKFS-fs (loop1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4567.655587] Lustre: 268067:0:(osd_scrub.c:1831:osd_index_restore()) lustre-MDT0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 4567.663625] Lustre: 268067:0:(osd_scrub.c:1831:osd_index_restore()) lustre-MDT0001: restore index '0x1010000' with [0x200000003:0x5:0x0]: rc = 0 [ 4567.673375] Lustre: 268067:0:(osd_scrub.c:1831:osd_index_restore()) lustre-MDT0001: restore index '0x1010000-MDT0001' with [0x200000005:0x2:0x0]: rc = 0 [ 4567.684149] Lustre: 268067:0:(osd_scrub.c:1831:osd_index_restore()) lustre-MDT0001: restore index '0x10000-MDT0001' with [0x200000005:0x1:0x0]: rc = 0 [ 4567.692576] Lustre: 268067:0:(osd_scrub.c:1831:osd_index_restore()) lustre-MDT0001: restore index '0x2010000' with [0x200000003:0x6:0x0]: rc = 0 [ 4567.699594] Lustre: 268067:0:(osd_scrub.c:1831:osd_index_restore()) lustre-MDT0001: restore index '0x10000' with [0x200000003:0x4:0x0]: rc = 0 [ 4567.705877] Lustre: 268067:0:(osd_scrub.c:1831:osd_index_restore()) lustre-MDT0001: restore index '0x2010000-MDT0001' with [0x200000005:0x3:0x0]: rc = 0 [ 4567.718798] Lustre: Found index 1 for lustre-MDT0001, updating log [ 4567.826409] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4567.863605] Lustre: lustre-MDT0001: Aborting client recovery [ 4568.312269] loop2: detected capacity change from 0 to 204800000 [ 4568.364246] LDISKFS-fs (loop2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4568.401954] Lustre: 268148:0:(osd_scrub.c:1831:osd_index_restore()) lustre-OST0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 4568.406789] Lustre: 268148:0:(osd_scrub.c:1831:osd_index_restore()) lustre-OST0000: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 4568.412208] Lustre: 268148:0:(osd_scrub.c:1831:osd_index_restore()) lustre-OST0000: restore index '0x20000-OST0000' with [0x200000005:0x1:0x0]: rc = 0 [ 4568.417524] Lustre: 268148:0:(osd_scrub.c:1831:osd_index_restore()) lustre-OST0000: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 4568.423878] Lustre: 268148:0:(osd_scrub.c:1831:osd_index_restore()) lustre-OST0000: restore index '0x1020000-OST0000' with [0x200000005:0x2:0x0]: rc = 0 [ 4568.428915] Lustre: 268148:0:(osd_scrub.c:1831:osd_index_restore()) lustre-OST0000: restore index '0x2020000-OST0000' with [0x200000005:0x3:0x0]: rc = 0 [ 4568.434871] Lustre: 268148:0:(osd_scrub.c:1831:osd_index_restore()) lustre-OST0000: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 4568.505181] Lustre: Found index 0 for lustre-OST0000, updating log [ 4568.574109] Lustre: lustre-OST0000: Aborting recovery [ 4568.990723] loop3: detected capacity change from 0 to 204800000 [ 4569.025164] LDISKFS-fs (loop3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4569.068712] Lustre: 268232:0:(osd_scrub.c:1831:osd_index_restore()) lustre-OST0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 4569.076954] Lustre: 268232:0:(osd_scrub.c:1831:osd_index_restore()) lustre-OST0001: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 4569.087434] Lustre: 268232:0:(osd_scrub.c:1831:osd_index_restore()) lustre-OST0001: restore index '0x1020000-OST0001' with [0x200000005:0x2:0x0]: rc = 0 [ 4569.096357] Lustre: 268232:0:(osd_scrub.c:1831:osd_index_restore()) lustre-OST0001: restore index '0x20000-OST0001' with [0x200000005:0x1:0x0]: rc = 0 [ 4569.103811] Lustre: 268232:0:(osd_scrub.c:1831:osd_index_restore()) lustre-OST0001: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 4569.111376] Lustre: 268232:0:(osd_scrub.c:1831:osd_index_restore()) lustre-OST0001: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 4569.120993] Lustre: 268232:0:(osd_scrub.c:1831:osd_index_restore()) lustre-OST0001: restore index '0x2020000-OST0001' with [0x200000005:0x3:0x0]: rc = 0 [ 4569.183221] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 4569.186264] Lustre: Skipped 1 previous similar message [ 4569.198408] Lustre: lustre-OST0001: Aborting recovery [ 4574.700142] LustreError: 268010:0:(lod_dev.c:510:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 8, retries 0, failed: rc = -108 [ 4576.741877] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 4576.743907] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 4576.757908] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000bd0 [ 4576.762468] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000bd0 [ 4577.768735] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:5 to 0x280000400:33) [ 4577.769258] Lustre: lustre-OST0001: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:5 to 0x2c0000400:33) [ 4580.377128] LustreError: 267977:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0000: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0000 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0000/identity_upcall, time 625us: rc = -2 [ 4593.121280] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4593.122313] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4593.129123] Lustre: Skipped 2 previous similar messages [ 4593.132886] Lustre: Skipped 1 previous similar message [ 4598.055986] Lustre: server umount lustre-MDT0000 complete [ 4598.240855] LustreError: 268173:0:(ldlm_lib.c:1178:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4598.248984] LustreError: 268173:0:(ldlm_lib.c:1178:target_handle_connect()) Skipped 5 previous similar messages [ 4598.506508] LustreError: 268074:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1773106782 with bad export cookie 2069977247855862285 [ 4598.509638] LustreError: MGC192.168.201.147@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4598.512748] LustreError: 268074:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4603.360552] LustreError: 267974:0:(ldlm_lib.c:1178:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4603.360644] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4603.361703] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 4603.361708] Lustre: Skipped 2 previous similar messages [ 4603.370482] LustreError: 267974:0:(ldlm_lib.c:1178:target_handle_connect()) Skipped 1 previous similar message [ 4603.375922] Lustre: Skipped 2 previous similar messages [ 4604.825111] Lustre: server umount lustre-MDT0001 complete [ 4611.347968] Lustre: server umount lustre-OST0000 complete [ 4625.887152] Lustre: lustre-OST0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 4625.980179] Lustre: server umount lustre-OST0001 complete [ 4629.339838] Lustre: DEBUG MARKER: == conf-sanity test 109a: test lctl clear_conf fsname ==== 21:40:13 (1773106813) [ 4640.641370] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing set_hostid [ 4644.805150] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing load_modules_local [ 4660.194078] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4660.310875] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 4660.326048] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 4660.371792] Lustre: lustre-MDT0000: new disk, initializing [ 4660.410686] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4660.419598] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 4661.986731] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing set_default_debug -1 all [ 4666.847813] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4666.895828] Lustre: 272272:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 4666.924938] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 4666.928237] Lustre: Skipped 1 previous similar message [ 4666.970506] Lustre: lustre-MDT0001: new disk, initializing [ 4667.015259] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4667.030085] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 4667.034209] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 4668.718432] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing set_default_debug -1 all [ 4672.376467] Lustre: DEBUG MARKER: oleg147-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 4673.949984] Lustre: DEBUG MARKER: oleg147-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 4676.924114] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4677.054375] Lustre: lustre-OST0000: new disk, initializing [ 4677.057239] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 4677.093963] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4679.396246] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing set_default_debug -1 all [ 4682.862537] Lustre: DEBUG MARKER: oleg147-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4686.330811] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 4686.335803] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 4686.379037] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 4690.986950] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [ 4695.156852] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [ 4698.582536] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [ 4705.218085] Lustre: Modifying parameter lustre-client.llite.max_read_ahead_mb=64 in log lustre-client [ 4747.745645] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4747.747629] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4747.749332] LustreError: Skipped 1 previous similar message [ 4747.750155] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4747.756594] Lustre: Skipped 1 previous similar message [ 4747.759207] Lustre: Skipped 1 previous similar message [ 4752.867103] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4752.869896] Lustre: Skipped 1 previous similar message [ 4757.986061] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4757.989027] Lustre: Skipped 1 previous similar message [ 4759.007155] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 4759.079290] Lustre: server umount lustre-OST0000 complete [ 4763.104868] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4763.111719] Lustre: Skipped 1 previous similar message [ 4766.777602] Lustre: server umount lustre-MDT0000 complete [ 4768.224688] LustreError: 272284:0:(ldlm_lib.c:1178:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4768.230517] LustreError: 272284:0:(ldlm_lib.c:1178:target_handle_connect()) Skipped 1 previous similar message [ 4768.562263] LustreError: 273050:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1773106952 with bad export cookie 2069977247855864987 [ 4768.566369] LustreError: MGC192.168.201.147@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4768.567730] LustreError: 273050:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4768.708897] Lustre: server umount lustre-MDT0001 complete [ 4772.268158] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4774.153490] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing set_default_debug -1 all [ 4776.106736] Lustre: server umount MGS complete [ 4780.967191] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4781.150213] LustreError: 277652:0:(ldlm_lib.c:1178:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4781.159509] LustreError: 277665:0:(qmt_pool.c:1614:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [ 4781.188699] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4782.988715] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing set_default_debug -1 all [ 4786.512820] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4788.334639] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing set_default_debug -1 all [ 4790.607320] Lustre: DEBUG MARKER: oleg147-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 4792.160245] Lustre: DEBUG MARKER: oleg147-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 4794.924825] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4797.290042] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing set_default_debug -1 all [ 4799.553404] Lustre: DEBUG MARKER: oleg147-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4837.344688] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4837.345349] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4837.347788] LustreError: Skipped 1 previous similar message [ 4837.348315] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4837.353019] Lustre: Skipped 1 previous similar message [ 4837.355082] Lustre: Skipped 2 previous similar messages [ 4839.083871] Lustre: server umount lustre-OST0000 complete [ 4846.826396] Lustre: server umount lustre-MDT0000 complete [ 4847.584632] LustreError: 278363:0:(ldlm_lib.c:1178:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4847.593070] LustreError: 278363:0:(ldlm_lib.c:1178:target_handle_connect()) Skipped 1 previous similar message [ 4848.667411] LustreError: 278370:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1773107032 with bad export cookie 2069977247855866919 [ 4848.671500] LustreError: MGC192.168.201.147@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4848.673405] LustreError: 278370:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4848.838684] Lustre: server umount lustre-MDT0001 complete [ 4853.774940] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing unload_modules_local [ 4855.169960] Key type lgssc unregistered [ 4855.350564] LNet: 281263:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4855.354345] LNetError: 281263:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4855.361550] LNet: Removed LNI 192.168.201.147@tcp [ 4855.794140] Key type .llcrypt unregistered [ 4855.795898] Key type ._llcrypt unregistered [ 4872.673315] Key type ._llcrypt registered [ 4872.675875] Key type .llcrypt registered [ 4872.734410] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing set_hostid [ 4880.074250] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing load_modules_local [ 4880.558589] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4880.622381] alg: No test for adler32 (adler32-zlib) [ 4881.517090] Lustre: Lustre: Build Version: 2.17.50_236_gea5aa0c [ 4881.633241] LNet: Added LNI 192.168.201.147@tcp [8/256/0/180] [ 4883.231166] Key type lgssc registered [ 4883.723029] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4899.962882] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4899.970628] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4901.119170] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 4901.144963] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 4901.195434] Lustre: lustre-MDT0000: new disk, initializing [ 4901.256361] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4901.272722] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 4903.186631] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing set_default_debug -1 all [ 4908.379539] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4908.433984] Lustre: 285160:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 4908.464656] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 4908.467515] Lustre: Skipped 1 previous similar message [ 4908.503801] Lustre: lustre-MDT0001: Not available for connect from 0@lo (not set up) [ 4908.519894] Lustre: lustre-MDT0001: new disk, initializing [ 4908.565834] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4908.585272] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 4908.588934] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 4910.292249] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing set_default_debug -1 all [ 4914.060644] Lustre: DEBUG MARKER: oleg147-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 4915.557766] Lustre: DEBUG MARKER: oleg147-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 4918.570238] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4918.703979] Lustre: lustre-OST0000: new disk, initializing [ 4918.707112] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 4918.750913] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4921.249175] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing set_default_debug -1 all [ 4923.904291] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 4923.907320] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 4923.945986] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 4924.876718] Lustre: DEBUG MARKER: oleg147-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4928.725467] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [ 4931.532080] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [ 4937.902159] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [ 4944.619485] Lustre: Modifying parameter lustre-client.llite.max_read_ahead_mb=64 in log lustre-client [ 4985.315349] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4985.316299] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4985.317112] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4985.319779] Lustre: Skipped 1 previous similar message [ 4987.236734] Lustre: server umount lustre-OST0000 complete [ 4990.432519] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4990.432784] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4990.433880] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4990.433884] Lustre: Skipped 1 previous similar message [ 4990.435616] LustreError: Skipped 1 previous similar message [ 4990.448775] Lustre: Skipped 1 previous similar message [ 4995.552784] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4995.556593] Lustre: Skipped 2 previous similar messages [ 5000.674032] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5000.677053] Lustre: Skipped 1 previous similar message [ 5003.231658] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 5003.316274] Lustre: server umount lustre-MDT0000 complete [ 5004.912247] LustreError: 285153:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1773107189 with bad export cookie 10792995867690351467 [ 5004.915585] LustreError: MGC192.168.201.147@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5004.917222] LustreError: 285153:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 5005.057904] Lustre: server umount lustre-MDT0001 complete [ 5008.493602] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5010.014682] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing set_default_debug -1 all [ 5011.856110] Lustre: server umount MGS complete [ 5016.658831] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5016.870953] LustreError: 290582:0:(ldlm_lib.c:1178:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5016.881292] LustreError: 290582:0:(ldlm_lib.c:1178:target_handle_connect()) Skipped 3 previous similar messages [ 5016.884984] LustreError: 290595:0:(qmt_pool.c:1614:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [ 5016.919258] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5018.742852] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing set_default_debug -1 all [ 5022.176339] LustreError: 290583:0:(ldlm_lib.c:1178:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5022.625797] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5022.844963] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5024.493279] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing set_default_debug -1 all [ 5026.609402] Lustre: DEBUG MARKER: oleg147-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 5028.131769] Lustre: DEBUG MARKER: oleg147-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 5030.924461] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5031.073770] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5033.523743] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing set_default_debug -1 all [ 5035.780387] Lustre: DEBUG MARKER: oleg147-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 5068.258495] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5068.259923] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 5068.259940] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5068.263077] Lustre: Skipped 1 previous similar message [ 5074.065635] Lustre: server umount lustre-OST0000 complete [ 5078.496629] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5078.501593] Lustre: Skipped 1 previous similar message [ 5081.579823] Lustre: server umount lustre-MDT0000 complete [ 5083.099226] LustreError: 290564:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1773107267 with bad export cookie 10792995867690353308 [ 5083.103064] LustreError: MGC192.168.201.147@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5083.103595] LustreError: 290564:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 5083.228572] Lustre: server umount lustre-MDT0001 complete [ 5087.347778] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing unload_modules_local [ 5088.594018] Key type lgssc unregistered [ 5088.731353] LNet: 294192:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5088.735206] LNetError: 294192:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5088.745359] LNet: Removed LNI 192.168.201.147@tcp [ 5089.101989] Key type .llcrypt unregistered [ 5089.103649] Key type ._llcrypt unregistered [ 5097.292184] Key type ._llcrypt registered [ 5097.294016] Key type .llcrypt registered [ 5097.343269] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing set_hostid [ 5103.978387] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing load_modules_local [ 5104.441054] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5104.484729] alg: No test for adler32 (adler32-zlib) [ 5105.365097] Lustre: Lustre: Build Version: 2.17.50_236_gea5aa0c [ 5105.480314] LNet: Added LNI 192.168.201.147@tcp [8/256/0/180] [ 5107.071353] Key type lgssc registered [ 5107.526674] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5117.173871] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing load_modules_local [ 5123.122109] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5123.128781] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5124.262600] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 5124.277172] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 5124.321589] Lustre: lustre-MDT0000: new disk, initializing [ 5124.349663] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5124.357282] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 5126.017150] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing set_default_debug -1 all [ 5130.514784] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5130.636006] Lustre: lustre-OST0000: new disk, initializing [ 5130.639434] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 5130.642366] Lustre: Skipped 1 previous similar message [ 5130.680868] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5131.404419] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 5131.411581] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 5131.428156] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 5133.059898] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing set_default_debug -1 all [ 5377.301562] Lustre: server umount lustre-MDT0000 complete [ 5378.854307] LustreError: 297352:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1773107563 with bad export cookie 14001033276695518389 [ 5378.859592] LustreError: MGC192.168.201.147@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5389.356575] Lustre: server umount lustre-OST0000 complete [ 5398.395585] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing set_hostid [ 5402.115164] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing load_modules_local [ 5416.378813] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5416.476467] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 5416.487810] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 5416.525805] Lustre: lustre-MDT0000: new disk, initializing [ 5416.555739] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5416.564868] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 5418.164185] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing set_default_debug -1 all [ 5422.931810] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5422.984517] Lustre: 302331:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 5423.015884] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 5423.018116] Lustre: Skipped 1 previous similar message [ 5423.065889] Lustre: lustre-MDT0001: new disk, initializing [ 5423.108512] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5423.122950] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 5423.126650] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 5424.715623] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing set_default_debug -1 all [ 5428.314140] Lustre: DEBUG MARKER: oleg147-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 5429.709859] Lustre: DEBUG MARKER: oleg147-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 5432.486841] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5432.623344] Lustre: lustre-OST0000: new disk, initializing [ 5432.625608] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 5432.660312] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5435.123770] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing set_default_debug -1 all [ 5438.644884] Lustre: DEBUG MARKER: oleg147-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 5440.002904] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 5440.007850] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 5440.062274] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 5440.429671] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 5440.529555] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 5442.214298] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 5442.306280] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 5445.089781] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 5445.090402] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5445.093271] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5445.093541] LustreError: Skipped 1 previous similar message [ 5445.105026] Lustre: Skipped 1 previous similar message [ 5449.307316] Lustre: server umount lustre-OST0000 complete [ 5453.792234] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 5453.795692] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5453.802114] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5453.803859] Lustre: Skipped 1 previous similar message [ 5455.328574] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5455.328935] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5456.991309] Lustre: server umount lustre-MDT0000 complete [ 5458.329913] LustreError: 302326:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1773107642 with bad export cookie 14001033276698459166 [ 5458.334940] LustreError: MGC192.168.201.147@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5458.335682] LustreError: 302326:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 5458.492664] Lustre: server umount lustre-MDT0001 complete [ 5459.884098] Lustre: DEBUG MARKER: SKIP: conf-sanity test_111 skipping SLOW test 111 [ 5460.603836] Lustre: DEBUG MARKER: == conf-sanity test 112a: mount OST with no_create option ========================================================== 21:54:04 (1773107644) [ 5463.092283] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5463.279404] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5464.712721] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing set_default_debug -1 all [ 5467.931096] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5469.713295] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing set_default_debug -1 all [ 5471.776751] Lustre: DEBUG MARKER: oleg147-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 5473.171484] Lustre: DEBUG MARKER: oleg147-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 5475.949193] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5478.353252] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing set_default_debug -1 all [ 5480.713489] Lustre: DEBUG MARKER: oleg147-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 5483.604484] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5483.701475] Lustre: lustre-OST0001: new disk, initializing [ 5483.704181] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 5483.742452] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 5483.746301] Lustre: Skipped 2 previous similar messages [ 5486.312039] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing set_default_debug -1 all [ 5490.702155] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 5490.706820] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 5490.733594] LustreError: 309445:0:(osp_precreate.c:969:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [ 5490.738369] LustreError: 309445:0:(osp_precreate.c:969:osp_precreate_cleanup_orphans()) Skipped 1 previous similar message [ 5491.086417] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 5491.184760] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 5491.747617] LustreError: 309449:0:(osp_precreate.c:969:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [ 5492.712535] Lustre: DEBUG MARKER: oleg147-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff9e7bc4cbb800.ost_server_uuid 50 [ 5492.771510] LustreError: 309449:0:(osp_precreate.c:969:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [ 5492.777278] LustreError: 309449:0:(osp_precreate.c:969:osp_precreate_cleanup_orphans()) Skipped 1 previous similar message [ 5493.385022] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff9e7bc4cbb800.ost_server_uuid in FULL state after 0 sec [ 5494.817925] LustreError: 309449:0:(osp_precreate.c:969:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [ 5494.825194] LustreError: 309449:0:(osp_precreate.c:969:osp_precreate_cleanup_orphans()) Skipped 3 previous similar messages [ 5494.877586] Lustre: DEBUG MARKER: oleg147-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff9e7bc4cbb800.ost_server_uuid 50 [ 5495.520107] Lustre: DEBUG MARKER: osc.lustre-OST0001-osc-ffff9e7bc4cbb800.ost_server_uuid in FULL state after 0 sec [ 5496.873294] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 5509.679844] Lustre: lustre-OST0001: Not available for connect from 192.168.201.47@tcp (stopping) [ 5509.683543] Lustre: Skipped 1 previous similar message [ 5511.140868] Lustre: lustre-OST0001-osc-MDT0000: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5511.146148] Lustre: Skipped 1 previous similar message [ 5513.796810] Lustre: server umount lustre-OST0001 complete [ 5514.789890] LustreError: 308121:0:(ldlm_lib.c:1178:target_handle_connect()) lustre-OST0001: not available for connect from 192.168.201.47@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5514.797957] LustreError: 308121:0:(ldlm_lib.c:1178:target_handle_connect()) Skipped 2 previous similar messages [ 5516.259019] LustreError: 308121:0:(ldlm_lib.c:1178:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5516.260699] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5516.261635] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5516.261639] Lustre: Skipped 2 previous similar messages [ 5516.268369] LustreError: 308121:0:(ldlm_lib.c:1178:target_handle_connect()) Skipped 1 previous similar message [ 5516.274995] Lustre: Skipped 1 previous similar message [ 5517.975928] Lustre: server umount lustre-OST0000 complete [ 5521.703222] Lustre: server umount lustre-MDT0000 complete [ 5523.302359] LustreError: 306219:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1773107707 with bad export cookie 14001033276698460034 [ 5523.305784] LustreError: MGC192.168.201.147@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5523.310665] LustreError: 306219:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 5523.461107] Lustre: server umount lustre-MDT0001 complete [ 5528.197421] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing unload_modules_local [ 5529.512324] Key type lgssc unregistered [ 5529.645318] LNet: 311176:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5529.649548] LNetError: 311176:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5529.660556] LNet: Removed LNI 192.168.201.147@tcp [ 5530.081058] Key type .llcrypt unregistered [ 5530.083078] Key type ._llcrypt unregistered [ 5541.354774] Key type ._llcrypt registered [ 5541.355913] Key type .llcrypt registered [ 5541.408589] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing load_modules_local [ 5542.008601] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5542.019412] alg: No test for adler32 (adler32-zlib) [ 5542.894565] Lustre: Lustre: Build Version: 2.17.50_236_gea5aa0c [ 5542.997703] LNet: Added LNI 192.168.201.147@tcp [8/256/0/180] [ 5544.591333] Key type lgssc registered [ 5545.070312] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5548.491345] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5548.501106] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5549.735569] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5551.386760] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing set_default_debug -1 all [ 5553.731955] Lustre: DEBUG MARKER: oleg147-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 5556.805681] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5557.011489] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5558.731029] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing set_default_debug -1 all [ 5563.448614] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5563.628691] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5566.330688] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing set_default_debug -1 all [ 5566.694891] LustreError: 314103:0:(ldlm_lib.c:1178:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5566.701195] LustreError: 314103:0:(ldlm_lib.c:1178:target_handle_connect()) Skipped 2 previous similar messages [ 5568.829234] Lustre: DEBUG MARKER: oleg147-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 5571.900771] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5572.033982] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 5574.615073] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing set_default_debug -1 all [ 5575.149771] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:33) [ 5575.151948] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:65) [ 5577.005715] Lustre: DEBUG MARKER: oleg147-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid [ 5581.179894] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing wait_import_state (FULL|IDLE) os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [ 5581.295294] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 5664.225433] LustreError: lustre-OST0001-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 5664.230568] Lustre: lustre-OST0001-osc-MDT0001: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5664.237393] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 5667.299405] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 5667.300532] Lustre: lustre-OST0001-osc-MDT0000: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5667.311404] LustreError: lustre-OST0001-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 5668.426566] Lustre: server umount lustre-OST0001 complete [ 5672.420788] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 5672.421611] LustreError: 314104:0:(ldlm_lib.c:1178:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5672.423450] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5672.424112] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5672.424116] Lustre: Skipped 1 previous similar message [ 5672.439892] LustreError: 314104:0:(ldlm_lib.c:1178:target_handle_connect()) Skipped 1 previous similar message [ 5676.770470] Lustre: server umount lustre-OST0000 complete [ 5680.096656] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 5680.100046] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5680.105451] Lustre: Skipped 1 previous similar message [ 5680.108246] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5680.110517] Lustre: Skipped 1 previous similar message [ 5684.596925] Lustre: server umount lustre-MDT0000 complete [ 5686.186586] LustreError: 312537:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1773107870 with bad export cookie 11866333154389162661 [ 5686.191383] LustreError: MGC192.168.201.147@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5686.192634] LustreError: 312537:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 5686.359172] Lustre: server umount lustre-MDT0001 complete [ 5691.299236] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing unload_modules_local [ 5692.668629] Key type lgssc unregistered [ 5692.838268] LNet: 317110:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5692.841656] LNetError: 317110:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5692.853347] LNet: Removed LNI 192.168.201.147@tcp [ 5693.248462] Key type .llcrypt unregistered [ 5693.250642] Key type ._llcrypt unregistered [ 5715.672896] Key type ._llcrypt registered [ 5715.675673] Key type .llcrypt registered [ 5715.734829] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing load_modules_local [ 5716.192644] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5716.269440] alg: No test for adler32 (adler32-zlib) [ 5717.167736] Lustre: Lustre: Build Version: 2.17.50_236_gea5aa0c [ 5717.278856] LNet: Added LNI 192.168.201.147@tcp [8/256/0/180] [ 5718.887204] Key type lgssc registered [ 5719.530275] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5723.549596] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5723.559110] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5724.834694] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5726.635742] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing set_default_debug -1 all [ 5730.180319] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5730.405743] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5732.204693] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing set_default_debug -1 all [ 5733.578334] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 5736.607658] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5736.768715] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5739.094191] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing set_default_debug -1 all [ 5739.876790] LustreError: 320813:0:(ldlm_lib.c:1178:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5739.890365] LustreError: 320813:0:(ldlm_lib.c:1178:target_handle_connect()) Skipped 2 previous similar messages [ 5742.799373] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5742.909850] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 5745.469525] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing set_default_debug -1 all [ 5750.250876] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:97) [ 5750.251060] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:65) [ 5755.674098] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 5761.812616] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 5775.842375] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5775.846543] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5775.851248] Lustre: Skipped 3 previous similar messages [ 5775.854082] Lustre: Skipped 2 previous similar messages [ 5777.706976] Lustre: server umount lustre-MDT0000 complete [ 5779.358237] LustreError: 319340:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1773107963 with bad export cookie 5096107901951916153 [ 5779.362374] LustreError: MGC192.168.201.147@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5779.365033] LustreError: 319340:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 5779.511610] Lustre: server umount lustre-MDT0001 complete [ 5791.497329] Lustre: server umount lustre-OST0000 complete [ 5803.379969] Lustre: server umount lustre-OST0001 complete [ 5810.327430] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing load_modules_local [ 5814.071404] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5814.217732] LustreError: 324545:0:(ldlm_lib.c:1178:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5814.243545] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5815.717040] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing set_default_debug -1 all [ 5819.056077] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5820.577727] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing set_default_debug -1 all [ 5821.596616] Lustre: 325652:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 5824.199133] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5824.352913] LustreError: 326007:0:(ldlm_lib.c:1178:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5824.360790] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5824.365177] Lustre: Skipped 1 previous similar message [ 5826.232699] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:129) [ 5826.598746] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing set_default_debug -1 all [ 5829.283538] LustreError: 326031:0:(ldlm_lib.c:1178:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5829.294153] LustreError: 326031:0:(ldlm_lib.c:1178:target_handle_connect()) Skipped 1 previous similar message [ 5829.649670] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5831.613284] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:97) [ 5832.228786] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing set_default_debug -1 all [ 5835.995970] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 5837.372903] Lustre: 327496:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 5844.960583] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 5844.964393] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5844.971453] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5844.974801] Lustre: Skipped 1 previous similar message [ 5849.566820] Lustre: server umount lustre-MDT0000 complete [ 5850.593547] LustreError: 325259:0:(ldlm_lib.c:1178:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5850.600892] LustreError: 325259:0:(ldlm_lib.c:1178:target_handle_connect()) Skipped 3 previous similar messages [ 5850.894243] LustreError: 325271:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1773108035 with bad export cookie 5096107901951917840 [ 5850.897191] LustreError: MGC192.168.201.147@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5850.897892] LustreError: 325271:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 5851.024914] Lustre: server umount lustre-MDT0001 complete [ 5861.898159] Lustre: server umount lustre-OST0000 complete [ 5873.576901] Lustre: server umount lustre-OST0001 complete [ 5875.001782] Lustre: DEBUG MARKER: SKIP: conf-sanity test_114 skipping SLOW test 114 [ 5875.607495] Lustre: DEBUG MARKER: SKIP: conf-sanity test_115 skipping excluded test 115 [ 5876.275092] Lustre: DEBUG MARKER: == conf-sanity test 116: big size MDT support ============ 22:01:00 (1773108060) [ 5884.977662] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing load_modules_local [ 5888.617114] loop0: detected capacity change from 0 to 1099511627776 [ 5888.828109] SGI XFS with ACLs, security attributes, quota, no debug enabled [ 5888.832401] *** VALIDATE xfs *** [ 5888.840261] XFS (loop0): Mounting V5 Filesystem [ 5888.854166] XFS (loop0): Ending clean mount [ 5889.363678] loop1: detected capacity change from 0 to 18691697672192 [ 5913.596564] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing set_hostid [ 5917.093251] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing load_modules_local [ 5929.876766] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5929.976568] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 5929.988312] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 5930.024639] Lustre: lustre-MDT0000: new disk, initializing [ 5930.051585] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5930.053906] Lustre: Skipped 1 previous similar message [ 5930.059206] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 5931.474679] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing set_default_debug -1 all [ 5936.065149] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5936.110404] Lustre: 332570:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 5936.133735] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 5936.137096] Lustre: Skipped 1 previous similar message [ 5936.176262] Lustre: lustre-MDT0001: new disk, initializing [ 5936.225773] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 5936.230475] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 5937.744365] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing set_default_debug -1 all [ 5941.237568] Lustre: DEBUG MARKER: oleg147-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 5942.590633] Lustre: DEBUG MARKER: oleg147-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 5945.017598] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5945.114756] Lustre: lustre-OST0000: new disk, initializing [ 5945.116537] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 5946.363612] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 5946.369095] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 5946.405702] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 5947.034408] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing set_default_debug -1 all [ 5950.260057] Lustre: DEBUG MARKER: oleg147-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 5951.780411] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 5951.861610] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 5953.331101] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 5953.411138] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 5956.184300] Lustre: server umount lustre-OST0000 complete [ 5961.696728] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 5961.697026] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5961.697918] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5961.697923] Lustre: Skipped 4 previous similar messages [ 5961.705772] Lustre: Skipped 4 previous similar messages [ 5966.816708] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5966.818609] Lustre: Skipped 2 previous similar messages [ 5966.846530] Lustre: server umount lustre-MDT0000 complete [ 5968.076381] LustreError: 334312:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1773108152 with bad export cookie 5096107901951919394 [ 5968.080724] LustreError: 334312:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 5968.081762] LustreError: MGC192.168.201.147@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5978.659116] Lustre: server umount lustre-MDT0001 complete [ 5981.811815] XFS (loop0): Unmounting Filesystem [ 5983.630901] Lustre: DEBUG MARKER: == conf-sanity test 117: lctl get_param return errors properly ========================================================== 22:02:47 (1773108167) [ 5986.227627] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5986.396599] LustreError: 336721:0:(ldlm_lib.c:1178:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5986.420866] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5986.422898] Lustre: Skipped 2 previous similar messages [ 5987.893811] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing set_default_debug -1 all [ 5991.265969] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5992.960948] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing set_default_debug -1 all [ 5995.088177] Lustre: DEBUG MARKER: oleg147-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 5996.474175] Lustre: DEBUG MARKER: oleg147-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 5998.980547] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6001.152910] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing set_default_debug -1 all [ 6003.241663] Lustre: DEBUG MARKER: oleg147-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 6021.601027] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 6021.601758] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6021.603740] LustreError: Skipped 1 previous similar message [ 6021.604971] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6021.609272] Lustre: Skipped 1 previous similar message [ 6023.380250] Lustre: server umount lustre-OST0000 complete [ 6026.720786] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6031.840213] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6031.843608] Lustre: Skipped 4 previous similar messages [ 6039.007107] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 6039.087954] Lustre: server umount lustre-MDT0000 complete [ 6040.619392] LustreError: 336703:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1773108224 with bad export cookie 5096107901951920276 [ 6040.623070] LustreError: MGC192.168.201.147@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6040.626398] LustreError: 336703:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 6040.788822] Lustre: server umount lustre-MDT0001 complete [ 6045.134864] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing unload_modules_local [ 6046.425084] Key type lgssc unregistered [ 6046.567524] LNet: 340015:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 6046.572268] LNetError: 340015:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 6046.580602] LNet: Removed LNI 192.168.201.147@tcp [ 6046.969114] Key type .llcrypt unregistered [ 6046.970207] Key type ._llcrypt unregistered [ 6058.362152] Key type ._llcrypt registered [ 6058.363785] Key type .llcrypt registered [ 6058.405821] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing load_modules_local [ 6058.916463] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 6058.933130] alg: No test for adler32 (adler32-zlib) [ 6059.797915] Lustre: Lustre: Build Version: 2.17.50_236_gea5aa0c [ 6059.900424] LNet: Added LNI 192.168.201.147@tcp [8/256/0/180] [ 6061.495179] Key type lgssc registered [ 6061.889944] Lustre: Echo OBD driver; http://www.lustre.org/ [ 6064.833360] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 6064.839542] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6066.017757] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6067.484332] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing set_default_debug -1 all [ 6070.669848] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6070.829110] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 6072.449273] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing set_default_debug -1 all [ 6074.609993] Lustre: DEBUG MARKER: oleg147-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 6075.959180] Lustre: DEBUG MARKER: oleg147-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 6078.531104] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6078.681306] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 6080.929203] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing set_default_debug -1 all [ 6083.056710] Lustre: DEBUG MARKER: oleg147-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 6097.378939] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6097.385719] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 6097.389396] Lustre: Skipped 1 previous similar message [ 6098.992662] Lustre: lustre-MDT0001: Not available for connect from 192.168.201.47@tcp (stopping) [ 6102.498575] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 6102.501912] Lustre: Skipped 1 previous similar message [ 6102.557700] Lustre: server umount lustre-MDT0001 complete [ 6104.100695] LustreError: 342775:0:(ldlm_lib.c:1178:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.201.47@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6104.107200] LustreError: 342775:0:(ldlm_lib.c:1178:target_handle_connect()) Skipped 2 previous similar messages [ 6106.299656] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6106.337895] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [ 6106.339908] Lustre: Found index 1 for lustre-MDT0001, updating log [ 6106.355206] Lustre: 341538:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 6106.470133] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 6107.891666] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing set_default_debug -1 all [ 6111.717963] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 6115.815107] LustreError: 345032:0:(obd_mount.c:212:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [ 6115.819847] LustreError: 345032:0:(tgt_mount.c:667:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [ 6115.824421] LustreError: 341543:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [ 6115.843650] LustreError: 345035:0:(obd_config.c:2136:class_config_llog_handler()) MGC192.168.201.147@tcp: cfg command failed: rc = -17 [ 6115.846643] Lustre: cmd=cf001 0:lustre-MDT0001-osp-MDT0000 1:osp 2:lustre-MDT0000-mdtlov_UUID [ 6115.846643] [ 6411.306459] Lustre: lustre-MDT0001: Not available for connect from 192.168.201.47@tcp (stopping) [ 6413.793789] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [ 6413.794115] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6413.800914] Lustre: Skipped 2 previous similar messages [ 6416.419896] Lustre: lustre-MDT0001: Not available for connect from 192.168.201.47@tcp (stopping) [ 6416.421965] Lustre: Skipped 2 previous similar messages [ 6416.531931] Lustre: server umount lustre-MDT0001 complete [ 6418.912899] LustreError: 341551:0:(ldlm_lib.c:1178:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6418.920642] LustreError: 341551:0:(ldlm_lib.c:1178:target_handle_connect()) Skipped 1 previous similar message [ 6419.971802] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6420.006070] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [ 6420.008897] Lustre: Found index 1 for lustre-MDT0001, updating log [ 6420.021787] Lustre: 341538:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 6420.131866] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 6421.461699] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing set_default_debug -1 all [ 6425.573695] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 6429.670419] LustreError: 357589:0:(obd_mount.c:212:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [ 6429.672840] LustreError: 357589:0:(tgt_mount.c:667:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [ 6429.675626] LustreError: 341543:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [ 6429.677875] LustreError: 341543:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [ 6429.695351] LustreError: 357592:0:(obd_config.c:797:class_setup()) Device 8 already setup (type osp) [ 6429.697475] LustreError: 357592:0:(obd_config.c:902:class_detach()) OBD device 8 still set up [ 6429.699185] LustreError: 357592:0:(obd_config.c:2136:class_config_llog_handler()) MGC192.168.201.147@tcp: cfg command failed: rc = -17 [ 6429.701653] Lustre: cmd=cf003 0:lustre-MDT0001-osp-MDT0000 1:lustre-MDT0001_UUID 2:192.168.201.147@tcp [ 6429.701653] [ 6727.649285] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6727.650121] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [ 6727.655904] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 6727.661515] Lustre: Skipped 1 previous similar message [ 6730.507567] Lustre: server umount lustre-MDT0001 complete [ 6732.768742] LustreError: 341545:0:(ldlm_lib.c:1178:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6732.775030] LustreError: 341545:0:(ldlm_lib.c:1178:target_handle_connect()) Skipped 1 previous similar message [ 6733.678318] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6733.713693] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [ 6733.715773] Lustre: Found index 1 for lustre-MDT0001, updating log [ 6733.730422] Lustre: 341538:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 6733.834321] Lustre: lustre-MDT0001: Imperative Recovery enabled, recovery window shrunk from 60-180 down to 60-180 [ 6734.982461] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing set_default_debug -1 all [ 6738.917134] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 6743.012905] LustreError: 370247:0:(obd_mount.c:212:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [ 6743.015078] LustreError: 370247:0:(tgt_mount.c:667:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [ 6743.017464] LustreError: 341543:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [ 6743.019339] LustreError: 341543:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [ 6743.036239] LustreError: 370250:0:(lod_lov.c:212:lod_add_device()) lustre-MDT0001-osp-MDT0000: cannot connect to next dev lustre-MDT0001_UUID (-114) [ 6743.041689] LustreError: 370250:0:(obd_config.c:2136:class_config_llog_handler()) MGC192.168.201.147@tcp: cfg command failed: rc = -114 [ 6743.046517] Lustre: cmd=cf014 0:lustre-MDT0000-mdtlov 1:lustre-MDT0001_UUID 2:1 3:1 [ 6743.046517] [ 7051.233901] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7051.235962] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7051.239659] Lustre: Skipped 3 previous similar messages [ 7051.239958] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7051.242788] Lustre: Skipped 3 previous similar messages [ 7056.093358] Lustre: server umount lustre-MDT0000 complete [ 7056.353597] LustreError: 359200:0:(ldlm_lib.c:1178:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7056.359035] LustreError: 359200:0:(ldlm_lib.c:1178:target_handle_connect()) Skipped 2 previous similar messages [ 7057.278149] LustreError: 341531:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1773109241 with bad export cookie 11519905118874338793 [ 7057.281817] LustreError: MGC192.168.201.147@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7057.287780] LustreError: 341531:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 7061.471895] LustreError: 343704:0:(ldlm_lib.c:1178:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7061.472222] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7063.478932] Lustre: server umount lustre-MDT0001 complete [ 7064.803695] Lustre: server umount lustre-OST0000 complete [ 7067.853869] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing set_hostid [ 7071.129503] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing load_modules_local [ 7082.500826] Lustre: DEBUG MARKER: == conf-sanity test 120: cross-target rename should not create bad symlinks ========================================================== 22:21:06 (1773109266) [ 7084.634443] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7084.726378] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 7084.738581] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 7084.774419] Lustre: lustre-MDT0000: new disk, initializing [ 7084.797656] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7084.805315] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 7086.127356] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing set_default_debug -1 all [ 7090.316525] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7090.366430] Lustre: 384774:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 7090.388135] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 7090.389697] Lustre: Skipped 1 previous similar message [ 7090.427750] Lustre: lustre-MDT0001: new disk, initializing [ 7090.456286] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7090.466443] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 7090.469716] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 7091.722734] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing set_default_debug -1 all [ 7094.736613] Lustre: DEBUG MARKER: oleg147-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 7095.874177] Lustre: DEBUG MARKER: oleg147-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 7098.074486] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7098.179834] Lustre: lustre-OST0000: new disk, initializing [ 7098.181591] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 7098.209086] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7100.139453] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing set_default_debug -1 all [ 7101.049730] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 7101.053035] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 7101.082241] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 7103.220283] Lustre: DEBUG MARKER: oleg147-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 7106.015912] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7106.018024] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7106.021553] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7106.023209] Lustre: Skipped 2 previous similar messages [ 7111.903996] Lustre: server umount lustre-MDT0000 complete [ 7113.066262] LustreError: 384768:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1773109297 with bad export cookie 11519905118874340725 [ 7113.068609] LustreError: 385564:0:(osp_object.c:617:osp_attr_get()) lustre-MDT0000-osp-MDT0001: osp_attr_get update error [0x200000401:0x2:0x0]: rc = -5 [ 7113.070127] LustreError: MGC192.168.201.147@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7113.071521] LustreError: 384768:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 7113.200908] Lustre: server umount lustre-MDT0001 complete [ 7124.630988] Lustre: server umount lustre-OST0000 complete [ 7133.759708] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing set_hostid [ 7136.923357] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing load_modules_local [ 7147.438952] Lustre: DEBUG MARKER: == conf-sanity test complete, duration 6914 sec ========== 22:22:11 (1773109331) [ 7147.998811] Lustre: DEBUG MARKER: === conf-sanity: start cleanup 22:22:12 (1773109332) === [ 7149.185319] Lustre: DEBUG MARKER: === conf-sanity: finish cleanup 22:22:13 (1773109333) === [ 7162.288788] Lustre: DEBUG MARKER: oleg147-server.virtnet: executing unload_modules_local [ 7163.417609] Key type lgssc unregistered [ 7163.555361] LNet: 392704:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 7163.559334] LNetError: 392704:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 7163.567336] LNet: Removed LNI 192.168.201.147@tcp [ 7163.884983] Key type .llcrypt unregistered [ 7163.886222] Key type ._llcrypt unregistered