[ 0.000000] Linux version 4.18.0rh8.10-debug (green@maintenance) (gcc version 8.5.0 20210514 (Red Hat 8.5.0-26) (GCC)) #2 SMP Mon Jul 14 01:24:22 EDT 2025 [ 0.000000] Command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers' [ 0.000000] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256 [ 0.000000] x86/fpu: Enabled xstate features 0x7, context size is 832 bytes, using 'standard' format. [ 0.000000] signal: max sigframe size: 1776 [ 0.000000] BIOS-provided physical RAM map: [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x00000000bffcdfff] usable [ 0.000000] BIOS-e820: [mem 0x00000000bffce000-0x00000000bfffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x0000000146dfffff] usable [ 0.000000] NX (Execute Disable) protection: active [ 0.000000] SMBIOS 2.8 present. [ 0.000000] DMI: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 0.000000] Hypervisor detected: KVM [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00 [ 0.000000] kvm-clock: using sched offset of 494068456 cycles [ 0.000000] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns [ 0.000000] tsc: Detected 2399.998 MHz processor [ 0.000000] last_pfn = 0x146e00 max_arch_pfn = 0x400000000 [ 0.000000] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT [ 0.000000] last_pfn = 0xbffce max_arch_pfn = 0x400000000 [ 0.000000] found SMP MP-table at [mem 0x000f54b0-0x000f54bf] [ 0.000000] RAMDISK: [mem 0xbcc54000-0xbffbffff] [ 0.000000] ACPI: Early table checksum verification disabled [ 0.000000] ACPI: RSDP 0x00000000000F52D0 000014 (v00 BOCHS ) [ 0.000000] ACPI: RSDT 0x00000000BFFE247C 000034 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACP 0x00000000BFFE2318 000074 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: DSDT 0x00000000BFFE0040 0022D8 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACS 0x00000000BFFE0000 000040 [ 0.000000] ACPI: APIC 0x00000000BFFE238C 000090 (v03 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: HPET 0x00000000BFFE241C 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: WAET 0x00000000BFFE2454 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: Reserving FACP table memory at [mem 0xbffe2318-0xbffe238b] [ 0.000000] ACPI: Reserving DSDT table memory at [mem 0xbffe0040-0xbffe2317] [ 0.000000] ACPI: Reserving FACS table memory at [mem 0xbffe0000-0xbffe003f] [ 0.000000] ACPI: Reserving APIC table memory at [mem 0xbffe238c-0xbffe241b] [ 0.000000] ACPI: Reserving HPET table memory at [mem 0xbffe241c-0xbffe2453] [ 0.000000] ACPI: Reserving WAET table memory at [mem 0xbffe2454-0xbffe247b] [ 0.000000] No NUMA configuration found [ 0.000000] Faking a node at [mem 0x0000000000000000-0x0000000146dfffff] [ 0.000000] NODE_DATA(0) allocated [mem 0x1465a3000-0x1465cdfff] [ 0.000000] Reserving 256MB of memory at 2752MB for crashkernel (System RAM: 4205MB) [ 0.000000] Zone ranges: [ 0.000000] DMA [mem 0x0000000000001000-0x0000000000ffffff] [ 0.000000] DMA32 [mem 0x0000000001000000-0x00000000ffffffff] [ 0.000000] Normal [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Device empty [ 0.000000] Movable zone start for each node [ 0.000000] Early memory node ranges [ 0.000000] node 0: [mem 0x0000000000001000-0x000000000009efff] [ 0.000000] node 0: [mem 0x0000000000100000-0x00000000bffcdfff] [ 0.000000] node 0: [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Zeroed struct page in unavailable ranges: 4756 pages [ 0.000000] Initmem setup node 0 [mem 0x0000000000001000-0x0000000146dfffff] [ 0.000000] ACPI: PM-Timer IO Port: 0x608 [ 0.000000] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1]) [ 0.000000] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23 [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level) [ 0.000000] Using ACPI (MADT) for SMP configuration information [ 0.000000] ACPI: HPET id: 0x8086a201 base: 0xfed00000 [ 0.000000] TSC deadline timer available [ 0.000000] smpboot: Allowing 4 CPUs, 0 hotplug CPUs [ 0.000000] kvm-guest: KVM setup pv remote TLB flush [ 0.000000] kvm-guest: setup PV sched yield [ 0.000000] PM: Registered nosave memory: [mem 0x00000000-0x00000fff] [ 0.000000] PM: Registered nosave memory: [mem 0x0009f000-0x0009ffff] [ 0.000000] PM: Registered nosave memory: [mem 0x000a0000-0x000effff] [ 0.000000] PM: Registered nosave memory: [mem 0x000f0000-0x000fffff] [ 0.000000] PM: Registered nosave memory: [mem 0xbffce000-0xbfffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xc0000000-0xfeffbfff] [ 0.000000] PM: Registered nosave memory: [mem 0xfeffc000-0xfeffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xff000000-0xfffbffff] [ 0.000000] PM: Registered nosave memory: [mem 0xfffc0000-0xffffffff] [ 0.000000] [mem 0xc0000000-0xfeffbfff] available for PCI devices [ 0.000000] Booting paravirtualized kernel on KVM [ 0.000000] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns [ 0.000000] setup_percpu: NR_CPUS:8192 nr_cpumask_bits:4 nr_cpu_ids:4 nr_node_ids:1 [ 0.000000] percpu: Embedded 63 pages/cpu s221184 r8192 d28672 u524288 [ 0.000000] kvm-guest: PV spinlocks enabled [ 0.000000] PV qspinlock hash table entries: 256 (order: 0, 4096 bytes, linear) [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 1059606 [ 0.000000] Policy zone: Normal [ 0.000000] Kernel command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] Specific versions of hardware are certified with Red Hat Enterprise Linux 8. Please see the list of hardware certified with Red Hat Enterprise Linux 8 at https://catalog.redhat.com. [ 0.000000] audit: disabled (until reboot) [ 0.000000] software IO TLB: area num 4. [ 0.000000] Memory: 2829652K/4306352K available (18435K kernel code, 11221K rwdata, 7248K rodata, 2908K init, 18040K bss, 524580K reserved, 0K cma-reserved) [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1 [ 0.000000] kmemleak: Kernel memory leak detector disabled [ 0.000000] ftrace: allocating 41240 entries in 162 pages [ 0.000000] ftrace: allocated 162 pages with 3 groups [ 0.000000] rcu: Hierarchical RCU implementation. [ 0.000000] rcu: RCU event tracing is enabled. [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=8192 to nr_cpu_ids=4. [ 0.000000] rcu: RCU callback double-/use-after-free debug enabled. [ 0.000000] Rude variant of Tasks RCU enabled. [ 0.000000] Tracing variant of Tasks RCU enabled. [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies. [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4 [ 0.000000] NR_IRQS: 524544, nr_irqs: 456, preallocated irqs: 16 [ 0.000000] random: get_random_bytes called from start_kernel+0x622/0x9a8 with crng_init=0 [ 0.001000] Console: colour *CGA 80x25 [ 0.001000] printk: console [ttyS1] enabled [ 0.001000] ACPI: Core revision 20220331 [ 0.001000] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns [ 0.001011] APIC: Switch to symmetric I/O mode setup [ 0.002265] x2apic enabled [ 0.003004] Switched APIC routing to physical x2apic. [ 0.004008] kvm-guest: setup PV IPIs [ 0.006587] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1 [ 0.007000] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 0.007026] Calibrating delay loop (skipped) preset value.. 4799.99 BogoMIPS (lpj=2399998) [ 0.008015] pid_max: default: 32768 minimum: 301 [ 0.009157] LSM: Security Framework initializing [ 0.010054] Yama: becoming mindful. [ 0.012007] SELinux: Initializing. [ 0.013079] *** VALIDATE selinux *** [ 0.022204] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, vmalloc) [ 0.026615] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, vmalloc) [ 0.027132] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.028106] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.029091] *** VALIDATE tmpfs *** [ 0.030363] *** VALIDATE proc *** [ 0.031212] *** VALIDATE cgroup *** [ 0.032006] *** VALIDATE cgroup2 *** [ 0.033260] x86/cpu: User Mode Instruction Prevention (UMIP) activated [ 0.034158] Last level iTLB entries: 4KB 0, 2MB 0, 4MB 0 [ 0.035009] Last level dTLB entries: 4KB 0, 2MB 0, 4MB 0, 1GB 0 [ 0.036030] Spectre V2 : User space: Vulnerable [ 0.037006] Speculative Store Bypass: Vulnerable [ 0.039865] debug: unmapping init [mem 0xffffffff97659000-0xffffffff97660fff] [ 0.041218] smpboot: CPU0: Intel(R) Xeon(R) CPU E5-2695 v2 @ 2.40GHz (family: 0x6, model: 0x3e, stepping: 0x4) [ 0.042755] Performance Events: IvyBridge events, full-width counters, Intel PMU driver. [ 0.043027] ... version: 2 [ 0.044015] ... bit width: 48 [ 0.045010] ... generic registers: 4 [ 0.046016] ... value mask: 0000ffffffffffff [ 0.047014] ... max period: 00007fffffffffff [ 0.048020] ... fixed-purpose events: 3 [ 0.049013] ... event mask: 000000070000000f [ 0.050318] rcu: Hierarchical SRCU implementation. [ 0.052492] smp: Bringing up secondary CPUs ... [ 0.053608] x86: Booting SMP configuration: [ 0.054030] .... node #0, CPUs: #1 #2 #3 [ 0.060013] smp: Brought up 1 node, 4 CPUs [ 0.062020] smpboot: Max logical packages: 1 [ 0.063012] smpboot: Total of 4 processors activated (19199.98 BogoMIPS) [ 0.148024] node 0 deferred pages initialised in 83ms [ 0.152336] devtmpfs: initialized [ 0.154359] x86/mm: Memory block size: 128MB [ 0.157461] gcov: version magic: 0x41383552 [ 0.161358] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns [ 0.162102] futex hash table entries: 1024 (order: 4, 65536 bytes, vmalloc) [ 0.163292] pinctrl core: initialized pinctrl subsystem [ 0.164222] [ 0.164833] ************************************************************* [ 0.165020] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.166012] ** ** [ 0.167013] ** IOMMU DebugFS SUPPORT HAS BEEN ENABLED IN THIS KERNEL ** [ 0.168013] ** ** [ 0.169015] ** This means that this kernel is built to expose internal ** [ 0.170012] ** IOMMU data structures, which may compromise security on ** [ 0.171014] ** your system. ** [ 0.172014] ** ** [ 0.173013] ** If you see this message and you are not debugging the ** [ 0.174019] ** kernel, report this immediately to your vendor! ** [ 0.175016] ** ** [ 0.176013] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.177013] ************************************************************* [ 0.178642] NET: Registered protocol family 16 [ 0.179499] DMA: preallocated 512 KiB GFP_KERNEL pool for atomic allocations [ 0.180062] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations [ 0.181065] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations [ 0.182502] cpuidle: using governor menu [ 0.185213] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5 [ 0.187572] PCI: Using configuration type 1 for base access [ 0.190129] core: PMU erratum BJ122, BV98, HSD29 worked around, HT is on [ 0.200135] HugeTLB registered 1.00 GiB page size, pre-allocated 0 pages [ 0.202102] HugeTLB registered 2.00 MiB page size, pre-allocated 0 pages [ 0.205113] cryptd: max_cpu_qlen set to 1000 [ 0.209796] ACPI: Added _OSI(Module Device) [ 0.211014] ACPI: Added _OSI(Processor Device) [ 0.212015] ACPI: Added _OSI(3.0 _SCP Extensions) [ 0.214012] ACPI: Added _OSI(Processor Aggregator Device) [ 0.218268] ACPI: 1 ACPI AML tables successfully acquired and loaded [ 0.224624] ACPI: Interpreter enabled [ 0.226073] ACPI: PM: (supports S0 S3 S4 S5) [ 0.228017] ACPI: Using IOAPIC for interrupt routing [ 0.230138] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug [ 0.233378] ACPI: Enabled 2 GPEs in block 00 to 0F [ 0.244585] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff]) [ 0.247079] acpi PNP0A03:00: _OSC: OS supports [ASPM ClockPM Segments MSI HPX-Type3] [ 0.250027] acpi PNP0A03:00: _OSC: not requesting OS control; OS requires [ExtendedConfig ASPM ClockPM MSI] [ 0.254146] acpi PNP0A03:00: fail to add MMCONFIG information, can't access extended PCI configuration space under this bridge. [ 0.260735] acpiphp: Slot [2] registered [ 0.262121] acpiphp: Slot [5] registered [ 0.264120] acpiphp: Slot [6] registered [ 0.265111] acpiphp: Slot [7] registered [ 0.267136] acpiphp: Slot [8] registered [ 0.268085] acpiphp: Slot [9] registered [ 0.269050] acpiphp: Slot [10] registered [ 0.271113] acpiphp: Slot [3] registered [ 0.272091] acpiphp: Slot [4] registered [ 0.274111] acpiphp: Slot [11] registered [ 0.275118] acpiphp: Slot [12] registered [ 0.277109] acpiphp: Slot [13] registered [ 0.279096] acpiphp: Slot [14] registered [ 0.280094] acpiphp: Slot [15] registered [ 0.281106] acpiphp: Slot [16] registered [ 0.283092] acpiphp: Slot [17] registered [ 0.284109] acpiphp: Slot [18] registered [ 0.286182] acpiphp: Slot [19] registered [ 0.287088] acpiphp: Slot [20] registered [ 0.289100] acpiphp: Slot [21] registered [ 0.290101] acpiphp: Slot [22] registered [ 0.291091] acpiphp: Slot [23] registered [ 0.293114] acpiphp: Slot [24] registered [ 0.295112] acpiphp: Slot [25] registered [ 0.296101] acpiphp: Slot [26] registered [ 0.298123] acpiphp: Slot [27] registered [ 0.300119] acpiphp: Slot [28] registered [ 0.302114] acpiphp: Slot [29] registered [ 0.303106] acpiphp: Slot [30] registered [ 0.305172] acpiphp: Slot [31] registered [ 0.307067] PCI host bridge to bus 0000:00 [ 0.309035] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window] [ 0.311022] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window] [ 0.314026] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window] [ 0.317026] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window] [ 0.320040] pci_bus 0000:00: root bus resource [mem 0xe0000000000-0xe007fffffff window] [ 0.323028] pci_bus 0000:00: root bus resource [bus 00-ff] [ 0.325182] pci 0000:00:00.0: [8086:1237] type 00 class 0x060000 [ 0.328009] pci 0000:00:01.0: [8086:7000] type 00 class 0x060100 [ 0.332243] pci 0000:00:01.1: [8086:7010] type 00 class 0x010180 [ 0.341792] pci 0000:00:01.1: reg 0x20: [io 0xc320-0xc32f] [ 0.346552] pci 0000:00:01.1: legacy IDE quirk: reg 0x10: [io 0x01f0-0x01f7] [ 0.349019] pci 0000:00:01.1: legacy IDE quirk: reg 0x14: [io 0x03f6] [ 0.350011] pci 0000:00:01.1: legacy IDE quirk: reg 0x18: [io 0x0170-0x0177] [ 0.352016] pci 0000:00:01.1: legacy IDE quirk: reg 0x1c: [io 0x0376] [ 0.354498] pci 0000:00:01.3: [8086:7113] type 00 class 0x068000 [ 0.358818] pci 0000:00:01.3: quirk: [io 0x0600-0x063f] claimed by PIIX4 ACPI [ 0.361041] pci 0000:00:01.3: quirk: [io 0x0700-0x070f] claimed by PIIX4 SMB [ 0.364599] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 [ 0.369011] pci 0000:00:02.0: reg 0x10: [io 0xc300-0xc31f] [ 0.382014] pci 0000:00:02.0: reg 0x20: [mem 0xe0000000000-0xe0000003fff 64bit pref] [ 0.387020] pci 0000:00:02.0: reg 0x30: [mem 0xfeb80000-0xfebbffff pref] [ 0.391839] pci 0000:00:05.0: [1af4:1001] type 00 class 0x010000 [ 0.398014] pci 0000:00:05.0: reg 0x10: [io 0xc000-0xc07f] [ 0.406023] pci 0000:00:05.0: reg 0x14: [mem 0xfebc0000-0xfebc0fff] [ 0.425013] pci 0000:00:05.0: reg 0x20: [mem 0xe0000004000-0xe0000007fff 64bit pref] [ 0.435392] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 [ 0.443021] pci 0000:00:06.0: reg 0x10: [io 0xc080-0xc0ff] [ 0.451019] pci 0000:00:06.0: reg 0x14: [mem 0xfebc1000-0xfebc1fff] [ 0.467014] pci 0000:00:06.0: reg 0x20: [mem 0xe0000008000-0xe000000bfff 64bit pref] [ 0.476099] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 [ 0.480015] pci 0000:00:07.0: reg 0x10: [io 0xc100-0xc17f] [ 0.485014] pci 0000:00:07.0: reg 0x14: [mem 0xfebc2000-0xfebc2fff] [ 0.497017] pci 0000:00:07.0: reg 0x20: [mem 0xe000000c000-0xe000000ffff 64bit pref] [ 0.505911] pci 0000:00:08.0: [1af4:1001] type 00 class 0x010000 [ 0.517022] pci 0000:00:08.0: reg 0x10: [io 0xc180-0xc1ff] [ 0.524019] pci 0000:00:08.0: reg 0x14: [mem 0xfebc3000-0xfebc3fff] [ 0.548015] pci 0000:00:08.0: reg 0x20: [mem 0xe0000010000-0xe0000013fff 64bit pref] [ 0.555714] pci 0000:00:09.0: [1af4:1001] type 00 class 0x010000 [ 0.560013] pci 0000:00:09.0: reg 0x10: [io 0xc200-0xc27f] [ 0.564015] pci 0000:00:09.0: reg 0x14: [mem 0xfebc4000-0xfebc4fff] [ 0.580017] pci 0000:00:09.0: reg 0x20: [mem 0xe0000014000-0xe0000017fff 64bit pref] [ 0.587144] pci 0000:00:0a.0: [1af4:1001] type 00 class 0x010000 [ 0.594018] pci 0000:00:0a.0: reg 0x10: [io 0xc280-0xc2ff] [ 0.599014] pci 0000:00:0a.0: reg 0x14: [mem 0xfebc5000-0xfebc5fff] [ 0.615020] pci 0000:00:0a.0: reg 0x20: [mem 0xe0000018000-0xe000001bfff 64bit pref] [ 0.624851] ACPI: PCI: Interrupt link LNKA configured for IRQ 10 [ 0.626245] ACPI: PCI: Interrupt link LNKB configured for IRQ 10 [ 0.628354] ACPI: PCI: Interrupt link LNKC configured for IRQ 11 [ 0.630357] ACPI: PCI: Interrupt link LNKD configured for IRQ 11 [ 0.632160] ACPI: PCI: Interrupt link LNKS configured for IRQ 9 [ 0.635230] iommu: Default domain type: Passthrough [ 0.637447] SCSI subsystem initialized [ 0.639102] ACPI: bus type USB registered [ 0.640083] usbcore: registered new interface driver usbfs [ 0.641052] usbcore: registered new interface driver hub [ 0.643061] usbcore: registered new device driver usb [ 0.644129] pps_core: LinuxPPS API ver. 1 registered [ 0.645010] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti [ 0.647052] PTP clock support registered [ 0.649051] EDAC MC: Ver: 3.0.0 [ 0.650281] PCI: Using ACPI for IRQ routing [ 0.651630] NetLabel: Initializing [ 0.653012] NetLabel: domain hash size = 128 [ 0.654016] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO [ 0.655085] NetLabel: unlabeled traffic allowed by default [ 0.657229] vgaarb: loaded [ 0.658215] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0 [ 0.660008] hpet0: 3 comparators, 64-bit 100.000000 MHz counter [ 0.668511] clocksource: Switched to clocksource kvm-clock [ 0.760463] VFS: Disk quotas dquot_6.6.0 [ 0.761475] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) [ 0.762957] *** VALIDATE ramfs *** [ 0.763710] *** VALIDATE hugetlbfs *** [ 0.764686] pnp: PnP ACPI init [ 0.766195] pnp: PnP ACPI: found 6 devices [ 0.781175] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns [ 0.784492] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window] [ 0.785898] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window] [ 0.787377] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window] [ 0.788975] pci_bus 0000:00: resource 7 [mem 0xc0000000-0xfebfffff window] [ 0.790317] pci_bus 0000:00: resource 8 [mem 0xe0000000000-0xe007fffffff window] [ 0.792423] NET: Registered protocol family 2 [ 0.794858] IP idents hash table entries: 131072 (order: 8, 1048576 bytes, vmalloc) [ 0.798835] tcp_listen_portaddr_hash hash table entries: 4096 (order: 5, 163840 bytes, vmalloc) [ 0.801647] TCP established hash table entries: 65536 (order: 7, 524288 bytes, vmalloc) [ 0.807254] TCP bind hash table entries: 65536 (order: 9, 2097152 bytes, vmalloc) [ 0.811104] TCP: Hash tables configured (established 65536 bind 65536) [ 0.813241] MPTCP token hash table entries: 8192 (order: 6, 393216 bytes, vmalloc) [ 0.815490] UDP hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 0.817433] UDP-Lite hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 0.819708] NET: Registered protocol family 1 [ 0.821573] RPC: Registered named UNIX socket transport module. [ 0.822919] RPC: Registered udp transport module. [ 0.826039] RPC: Registered tcp transport module. [ 0.827926] RPC: Registered tcp NFSv4.1 backchannel transport module. [ 0.830157] NET: Registered protocol family 44 [ 0.831314] pci 0000:00:00.0: Limiting direct PCI/PCI transfers [ 0.833204] pci 0000:00:01.0: PIIX3: Enabling Passive Release [ 0.834642] pci 0000:00:01.0: Activating ISA DMA hang workarounds [ 0.836595] PCI: CLS 0 bytes, default 64 [ 0.837829] Unpacking initramfs... [ 2.145208] debug: unmapping init [mem 0xffff9337bcc54000-0xffff9337bffbffff] [ 2.148212] PCI-DMA: Using software bounce buffering for IO (SWIOTLB) [ 2.149884] software IO TLB: mapped [mem 0x00000000a8000000-0x00000000ac000000] (64MB) [ 2.153156] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 2.622242] Initialise system trusted keyrings [ 2.624478] Key type blacklist registered [ 2.626701] workingset: timestamp_bits=36 max_order=20 bucket_order=0 [ 2.637953] zbud: loaded [ 2.641844] *** VALIDATE nfs *** [ 2.643167] *** VALIDATE nfs4 *** [ 2.644882] pstore: using deflate compression [ 2.648366] Platform Keyring initialized [ 2.755270] NET: Registered protocol family 38 [ 2.757483] Key type asymmetric registered [ 2.758856] Asymmetric key parser 'x509' registered [ 2.760842] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 247) [ 2.763598] io scheduler mq-deadline registered [ 2.765466] io scheduler kyber registered [ 2.767268] io scheduler bfq registered [ 2.769458] atomic64_test: passed for x86-64 platform with CX8 and with SSE [ 2.772886] shpchp: Standard Hot Plug PCI Controller Driver version: 0.4 [ 2.776303] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0 [ 2.781206] ACPI: Power Button [PWRF] [ 2.787623] ACPI: \_SB_.LNKB: Enabled at IRQ 10 [ 2.792667] ACPI: \_SB_.LNKA: Enabled at IRQ 11 [ 2.811826] ACPI: \_SB_.LNKC: Enabled at IRQ 11 [ 2.818904] ACPI: \_SB_.LNKD: Enabled at IRQ 10 [ 2.852052] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled [ 2.882446] 00:03: ttyS1 at I/O 0x2f8 (irq = 3, base_baud = 115200) is a 16550A [ 2.909259] 00:04: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A [ 2.914108] Non-volatile memory driver v1.3 [ 2.915647] Linux agpgart interface v0.103 [ 2.946745] virtio_blk virtio1: [vda] 145896 512-byte logical blocks (74.7 MB/71.2 MiB) [ 2.949355] vda: detected capacity change from 0 to 74698752 [ 2.979804] virtio_blk virtio2: [vdb] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) [ 2.983174] vdb: detected capacity change from 0 to 1073741824 [ 3.006292] virtio_blk virtio3: [vdc] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 3.009682] vdc: detected capacity change from 0 to 2621440000 [ 3.030114] virtio_blk virtio4: [vdd] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 3.033152] vdd: detected capacity change from 0 to 2621440000 [ 3.059324] virtio_blk virtio5: [vde] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 3.062547] vde: detected capacity change from 0 to 4294967296 [ 3.077864] virtio_blk virtio6: [vdf] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 3.080911] vdf: detected capacity change from 0 to 4294967296 [ 3.088166] libphy: Fixed MDIO Bus: probed [ 3.095211] usbcore: registered new interface driver usbserial_generic [ 3.097972] usbserial: USB Serial support registered for generic [ 3.100946] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12 [ 3.105746] serio: i8042 KBD port at 0x60,0x64 irq 1 [ 3.108120] serio: i8042 AUX port at 0x60,0x64 irq 12 [ 3.111025] mousedev: PS/2 mouse device common for all mice [ 3.114682] rtc_cmos 00:05: RTC can wake from S4 [ 3.117083] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input1 [ 3.118614] rtc_cmos 00:05: registered as rtc0 [ 3.123530] rtc_cmos 00:05: alarms up to one day, y3k, 242 bytes nvram, hpet irqs [ 3.125166] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input4 [ 3.126705] intel_pstate: CPU model not supported [ 3.132939] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input3 [ 3.136325] hid: raw HID events driver (C) Jiri Kosina [ 3.139270] usbcore: registered new interface driver usbhid [ 3.141413] usbhid: USB HID core driver [ 3.143075] drop_monitor: Initializing network drop monitor service [ 3.144931] Initializing XFRM netlink socket [ 3.146478] NET: Registered protocol family 10 [ 3.148679] Segment Routing with IPv6 [ 3.150282] NET: Registered protocol family 17 [ 3.152863] mpls_gso: MPLS GSO support [ 3.158346] RAS: Correctable Errors collector initialized. [ 3.160711] AVX version of gcm_enc/dec engaged. [ 3.162531] AES CTR mode by8 optimization enabled [ 3.240890] sched_clock: Marking stable (3240866440, 0)->(4147911942, -907045502) [ 3.244809] registered taskstats version 1 [ 3.246978] Loading compiled-in X.509 certificates [ 3.249296] zswap: loaded using pool lzo/zbud [ 3.276985] Key type big_key registered [ 3.289780] Key type encrypted registered [ 3.291336] ima: No TPM chip found, activating TPM-bypass! [ 3.293698] ima: Allocated hash algorithm: sha1 [ 3.295624] ima: No architecture policies found [ 3.297601] evm: Initialising EVM extended attributes: [ 3.299532] evm: security.selinux [ 3.300846] evm: security.ima [ 3.302080] evm: security.capability [ 3.303558] evm: HMAC attrs: 0x1 [ 3.306146] rtc_cmos 00:05: setting system clock to 2026-09-03 13:02:38 UTC (1788440558) [ 3.313539] debug: unmapping init [mem 0xffffffff98603000-0xffffffff987fffff] [ 3.316944] debug: unmapping init [mem 0xffffffff97382000-0xffffffff97658fff] [ 3.324108] Write protecting the kernel read-only data: 28672k [ 3.329980] debug: unmapping init [mem 0xffffffff95a03000-0xffffffff95bfffff] [ 3.333225] debug: unmapping init [mem 0xffffffff96314000-0xffffffff963fffff] [ 3.369840] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 3.376733] systemd[1]: Detected virtualization kvm. [ 3.378814] systemd[1]: Detected architecture x86-64. [ 3.380877] systemd[1]: Running in initial RAM disk. Welcome to Rocky Linux 8.10 (Green Obsidian) dracut-049-233.git20240115.el8 (Initramfs)! [ 3.405064] systemd[1]: No hostname configured. [ 3.406866] systemd[1]: Set hostname to . [ 3.409066] random: systemd: uninitialized urandom read (16 bytes read) [ 3.411611] systemd[1]: Initializing machine ID from random generator. [ 3.548778] random: systemd: uninitialized urandom read (16 bytes read) [ 3.551635] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ 3.557975] random: systemd: uninitialized urandom read (16 bytes read) [ 3.560852] systemd[1]: Listening on udev Control Socket. [ OK ] Listening on udev Control Socket. [ 3.565896] systemd[1]: Reached target Swap. [ OK ] Reached target Swap. [ OK ] Listening on Journal Socket. [ OK ] Started Memstrack Anylazing Service. Starting Create list of required st…ce nodes for the current kernel... [ OK ] Listening on Journal Socket (/dev/log). Starting Journal Service... [ OK ] Reached target Timers. [ OK ] Reached target Local File Systems. Starting Create Volatile Files and Directories... [ OK ] Reached target Initrd Root Device. [ OK ] Reached target Paths. Starting Setup Virtual Console... [ OK ] Listening on udev Kernel Socket. [ OK ] Reached target Sockets. [ OK ] Reached target Local Encrypted Volumes. [ OK ] Reached target Slices. Starting Apply Kernel Variables... [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Started Create Volatile Files and Directories. [ OK ] Started Setup Virtual Console. Starting dracut cmdline hook... Starting Create Static Device Nodes in /dev... [ OK ] Started Apply Kernel Variables. [ OK ] Started Create Static Device Nodes in /dev. [ OK ] Started Journal Service. [ OK ] Started dracut cmdline hook. Starting dracut pre-udev hook... [ 4.156265] device-mapper: uevent: version 1.0.3 [ 4.158827] device-mapper: ioctl: 4.46.0-ioctl (2022-02-22) initialised: dm-devel@redhat.com [ OK ] Started dracut pre-udev hook. Starting udev Kernel Device Manager... [ OK ] Started udev Kernel Device Manager. [ 4.460325] random: fast init done Starting dracut pre-trigger hook... [ OK ] Started dracut pre-trigger hook. Starting udev Coldplug all Devices... Mounting Kernel Configuration File System... [ OK ] Mounted Kernel Configuration File System. [ OK ] Started udev Coldplug all Devices. [ OK ] Reached target System Initialization. [ OK ] Reached target Basic System. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. Starting dracut initqueue hook... [ 4.845329] virtio_net virtio0 ens2: renamed from eth0 [ 4.958133] scsi host0: ata_piix [ 5.015214] scsi host1: ata_piix [ 5.016778] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc320 irq 14 [ 5.019270] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc328 irq 15 [ 8.528042] dracut-initqueue[578]: RTNETLINK answers: File exists [ 9.828187] random: crng init done [ 9.829686] random: 7 urandom warning(s) missed due to ratelimiting Starting nbd nbd0... [ OK ] Started nbd nbd0. [ OK ] Started dracut initqueue hook. Mounting /sysroot... [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. [ 10.255716] EXT4-fs (nbd0): mounted filesystem with ordered data mode. Opts: (null) [ OK ] Mounted /sysroot. [ OK ] Reached target Initrd Root File System. Starting Reload Configuration from the Real Root... [ OK ] Started Reload Configuration from the Real Root. [ OK ] Reached target Initrd File Systems. [ OK ] Reached target Initrd Default Target. Starting dracut pre-pivot and cleanup hook... [ OK ] Started dracut pre-pivot and cleanup hook. Starting Cleaning Up and Shutting Down Daemons... [ OK ] Stopped dracut pre-pivot and cleanup hook. [ OK ] Stopped target Initrd Default Target. [ OK ] Stopped target Initrd Root Device. Stopping Hardware RNG Entropy Gatherer Daemon... [ OK ] Stopped target Timers. [ OK ] Stopped target Remote File Systems. [ OK ] Stopped target Remote File Systems (Pre). [ OK ] Stopped dracut initqueue hook. [ OK ] Stopped Hardware RNG Entropy Gatherer Daemon. [ OK ] Stopped target Basic System. [ OK ] Stopped target Sockets. [ OK ] Stopped target Slices. [ OK ] Stopped target Paths. [ OK ] Stopped target System Initialization. [ OK ] Stopped Apply Kernel Variables. [ OK ] Stopped udev Coldplug all Devices. [ OK ] Stopped dracut pre-trigger hook. [ OK ] Stopped Create Volatile Files and Directories. Stopping udev Kernel Device Manager... [ OK ] Stopped target Local File Systems. [ OK ] Stopped target Swap. [ OK ] Stopped target Local Encrypted Volumes. [ OK ] Stopped Dispatch Password Requests to Console Directory Watch. [ OK ] Stopped udev Kernel Device Manager. [ OK ] Started Cleaning Up and Shutting Down Daemons. [ OK ] Stopped Create Static Device Nodes in /dev. [ OK ] Stopped Create list of required sta…vice nodes for the current kernel. [ OK ] Stopped dracut pre-udev hook. [ OK ] Stopped dracut cmdline hook. [ OK ] Closed udev Control Socket. [ OK ] Closed udev Kernel Socket. Starting Cleanup udevd DB... [ OK ] Started Cleanup udevd DB. [ OK ] Reached target Switch Root. Starting Switch Root... [ 13.261691] printk: systemd: 26 output lines suppressed due to ratelimiting [ 14.020518] SELinux: Disabled at runtime. [ 14.145960] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 14.172543] systemd[1]: Detected virtualization kvm. [ 14.177841] systemd[1]: Detected architecture x86-64. Welcome to Rocky Linux 8.10 (Green Obsidian)! [ 15.922496] systemd[1]: initrd-switch-root.service: Succeeded. [ 15.932175] systemd[1]: Stopped Switch Root. [ OK ] Stopped Switch Root. [ 15.945490] systemd[1]: systemd-journald.service: Service has no hold-off time (RestartSec=0), scheduling restart. [ 15.957467] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. [ 15.971871] systemd[1]: Stopped Journal Service. [ OK ] Stopped Journal Service. [ 15.992555] systemd[1]: Starting Journal Service... Starting Journal Service... [ 16.009551] systemd[1]: Listening on udev Kernel Socket. [ OK ] Listening on udev Kernel Socket. Activating swap /dev/disk/by-label/SWAP... [ OK ] Listening on RPCbind Server Activation Socket. [ 16.064527] Adding 1048572k swap on /dev/vdb. Priority:-2 extents:1 across:1048572k FS [ OK ] Reached target rpc_pipefs.target. Mounting Huge Pages File System... Starting Apply Kernel Variables... [ OK ] Started Forward Password Requests to Wall Directory Watch. [ OK ] Reached target RPC Port Mapper. [ OK ] Created slice system-getty.slice. [ OK ] Created slice system-sshd\x2dkeygen.slice. Mounting Kernel Debug File System... [ OK ] Listening on initctl Compatibility Named Pipe. Starting Create list of required st…ce nodes for the current kernel... [ OK ] Stopped target Switch Root. [ OK ] Stopped target Initrd File Systems. Mounting POSIX Message Queue File System... [ OK ] Listening on udev Control Socket. Starting udev Coldplug all Devices... [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Reached target Paths. [ OK ] Reached target Local Encrypted Volumes. [ OK ] Created slice User and Session Slice. [ OK ] Reached target Slices. Starting Remount Root and Kernel File Systems... [ OK ] Listening on Process Core Dump Socket. [ OK ] Stopped target Initrd Root File System. [FAILED] Failed to set up automount Arbitrar…rmats File System Automount Point. See 'systemctl status proc-sys-fs-binfmt_misc.automount' for details. [ OK ] Created slice system-serial\x2dgetty.slice. [ OK ] Started Journal Service. [ OK ] Activated swap /dev/disk/by-label/SWAP. [ OK ] Mounted Huge Pages File System. [ OK ] Started Apply Kernel Variables. [ OK ] Mounted Kernel Debug File System. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Mounted POSIX Message Queue File System. [FAILED] Failed to start Remount Root and Kernel File Systems. See 'systemctl status systemd-remount-fs.service' for details. Starting Configure read-only root support... Starting Create Static Device Nodes in /dev... [ OK ] Reached target Swap. Starting Flush Journal to Persistent Storage... [ OK ] Started Create Static Device Nodes in /dev. [ OK ] Started udev Coldplug all Devices. [ OK ] Started Flush Journal to Persistent Storage. [ OK ] Reached target Local File Systems (Pre). Mounting /home/green/git/lustre-release... Mounting /mnt... Starting udev Kernel Device Manager... [ OK ] Mounted /mnt. [ 17.618507] squashfs: version 4.0 (2009/01/31) Phillip Lougher [ OK ] Mounted /home/green/git/lustre-release. [ OK ] Started udev Kernel Device Manager. [ 18.553973] input: PC Speaker as /devices/platform/pcspkr/input/input5 [ 18.654181] piix4_smbus 0000:00:01.3: SMBus Host Controller at 0x700, revision 0 [ 19.138611] RAPL PMU: API unit is 2^-32 Joules, 0 fixed counters, 10737418240 ms ovfl timer [ 19.264099] EDAC sbridge: Ver: 1.1.2 [* ] A start job is running for Configur…-only root support (7s / no limit) [** ] A start job is running for Configur…-only root support (8s / no limit)[ 24.127820] Key type dns_resolver registered [*** ] A start job is running for Configur…-only root support (8s / no limit)[ 25.033712] NFS: Registering the id_resolver key type [ 25.037516] Key type id_resolver registered [ 25.040718] Key type id_legacy registered [ *** ] A start job is running for Configur…-only root support (9s / no limit) [ OK ] Started Configure read-only root support. [ OK ] Reached target Local File Systems. Starting Rebuild Dynamic Linker Cache... Starting Mark the need to relabel after reboot... Starting Create Volatile Files and Directories... Starting Load/Save Random Seed... [ OK ] Started Mark the need to relabel after reboot. [ OK ] Started Load/Save Random Seed. [ OK ] Started Create Volatile Files and Directories. Starting RPC Bind... Starting Update UTMP about System Boot/Shutdown... [ OK ] Started Update UTMP about System Boot/Shutdown. [ OK ] Started RPC Bind. [ OK ] Started Rebuild Dynamic Linker Cache. Starting Update is Completed... [ OK ] Started Update is Completed. [ OK ] Reached target System Initialization. [ OK ] Listening on D-Bus System Message Bus Socket. [ OK ] Reached target Sockets. [ OK ] Reached target Basic System. Starting Login Service... [ OK ] Started Hardware RNG Entropy Gatherer Daemon. Starting Restore /run/initramfs on shutdown... [ OK ] Started daily update of the root trust anchor for DNSSEC. [ OK ] Started irqbalance daemon. [ OK ] Started dnf makecache --timer. [ OK ] Started D-Bus System Message Bus. Starting Network Manager... [ OK ] Started Daily Cleanup of Temporary Directories. [ OK ] Reached target Timers. [ OK ] Reached target sshd-keygen.target. [ OK ] Started Restore /run/initramfs on shutdown. [ OK ] Started Network Manager. [ OK ] Reached target Network. Starting OpenSSH server daemon... Starting Dynamic System Tuning Daemon... Starting GSSAPI Proxy Daemon... Starting Network Manager Wait Online... Starting Hostname Service... [ OK ] Started Login Service. [ OK ] Started OpenSSH server daemon. [ OK ] Started GSSAPI Proxy Daemon. [ OK ] Reached target NFS client services. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Starting Permit User Sessions... [ OK ] Started Permit User Sessions. [ OK ] Started Getty on tty1. [ OK ] Started Command Scheduler. [ OK ] Started Serial Getty on ttyS0. [ OK ] Started Serial Getty on ttyS1. [ OK ] Reached target Login Prompts. [ OK ] Started Hostname Service. Starting Network Manager Script Dispatcher Service... [ OK ] Started Network Manager Script Dispatcher Service. [ OK ] Started Network Manager Wait Online. [ OK ] Reached target Network is Online. Starting Crash recovery kernel arming... Starting Notify NFS peers of a restart... Starting System Logging Service... [ OK ] Started Notify NFS peers of a restart. [ OK ] Started System Logging Service. Rocky Linux 8.10 (Green Obsidian) Kernel 4.18.0rh8.10-debug on an x86_64 oleg248-server login: [ 54.058106] hrtimer: interrupt took 2116954 ns [ 92.737830] libcfs: loading out-of-tree module taints kernel. [ 92.813831] Key type ._llcrypt registered [ 92.823971] Key type .llcrypt registered [ 92.958162] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_hostid [ 113.636980] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing load_modules_local [ 116.087936] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 116.101916] alg: No test for adler32 (adler32-zlib) [ 117.501271] Lustre: Lustre: Build Version: 2.17.56_51_g2aa7cea [ 118.357953] LNet: Added LNI 192.168.202.148@tcp [8/256/0/180] [ 120.071165] Key type lgssc registered [ 122.800805] Lustre: Echo OBD driver; http://www.lustre.org/ [ 141.494396] ZFS: Loaded module v2.3.2-1, ZFS pool version 5000, ZFS filesystem version 5 [ 185.807209] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing load_modules_local [ 199.666348] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 199.698302] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 201.046805] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 201.095292] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 201.192450] Lustre: lustre-MDT0000: new disk, initializing [ 201.270955] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 201.291801] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 205.829988] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 220.387946] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 220.598206] Lustre: 6506:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 220.716540] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 220.731278] Lustre: Skipped 1 previous similar message [ 220.880528] Lustre: lustre-MDT0001: new disk, initializing [ 220.990426] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 221.012885] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 221.028832] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 226.027922] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 231.480435] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 242.321034] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 242.633867] Lustre: lustre-OST0000: new disk, initializing [ 242.645671] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 242.662079] Lustre: 8444:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 242.776946] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 244.503124] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 244.533224] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 244.669970] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 250.195275] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 265.856724] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 266.020691] Lustre: lustre-OST0001: new disk, initializing [ 266.029187] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 266.041570] Lustre: 9520:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 266.142361] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 271.393651] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 271.406675] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 271.511952] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 273.258876] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 286.803446] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 293.599645] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 298.584562] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing check_logdir /tmp/testlogs/ [ 304.644602] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing yml_node [ 309.462294] Lustre: DEBUG MARKER: Client: 2.17.56.51 [ 312.035800] Lustre: DEBUG MARKER: MDS: 2.17.56.51 [ 314.437209] Lustre: DEBUG MARKER: OSS: 2.17.56.51 [ 316.179643] Lustre: DEBUG MARKER: -----============= acceptance-small: conf-sanity ============----- Thu Sep 3 09:07:50 EDT 2026 [ 333.096195] Lustre: DEBUG MARKER: excepting tests: 102 106 115 32newtarball [ 335.127287] Lustre: DEBUG MARKER: skipping tests SLOW=no: 45 69 106 111 114 [ 353.248783] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 353.289846] Lustre: Skipped 1 previous similar message [ 353.301776] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 353.318911] Lustre: Skipped 3 previous similar messages [ 358.064330] Lustre: server umount lustre-MDT0000 complete [ 362.915705] LustreError: 9519:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788440918 with bad export cookie 6997162701422349524 [ 362.921379] LustreError: MGC192.168.202.148@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 362.924936] LustreError: 9519:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 363.490254] LustreError: 6513:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 363.493022] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 363.501238] LustreError: 6513:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 6 previous similar messages [ 363.514522] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 363.519695] Lustre: Skipped 3 previous similar messages [ 363.544554] Lustre: Skipped 1 previous similar message [ 367.538946] LustreError: 10345:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 367.546969] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 367.578092] LustreError: 10345:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 367.623427] Lustre: Skipped 1 previous similar message [ 372.704025] LustreError: 9222:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 372.705032] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 372.722540] LustreError: 9222:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 372.745109] Lustre: Skipped 1 previous similar message [ 377.312602] Lustre: lustre-MDT0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 377.710568] Lustre: server umount lustre-MDT0001 complete [ 382.098181] Lustre: server umount lustre-OST0000 complete [ 386.117358] Lustre: server umount lustre-OST0001 complete [ 393.389405] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_hostid [ 404.674808] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing load_modules_local [ 441.956848] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 442.287523] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 442.339409] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 442.470465] Lustre: lustre-MDT0000: new disk, initializing [ 442.612404] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 442.632375] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 447.752694] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 460.649235] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 460.869093] Lustre: 15073:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 461.002943] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 461.013357] Lustre: Skipped 1 previous similar message [ 461.166500] Lustre: lustre-MDT0001: new disk, initializing [ 461.352795] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 461.410396] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 461.429320] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 467.178955] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 477.565090] Lustre: DEBUG MARKER: oleg248-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 484.095866] Lustre: DEBUG MARKER: oleg248-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 491.911815] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 492.201366] Lustre: lustre-OST0000: new disk, initializing [ 492.205637] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 492.212295] Lustre: 17045:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 492.328729] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 494.041800] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 494.052924] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 494.190665] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 499.286939] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 510.232385] Lustre: DEBUG MARKER: oleg248-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 515.294894] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 515.572876] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 521.300987] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 521.623811] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 524.772134] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 524.780911] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 524.787403] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 524.811258] Lustre: Skipped 1 previous similar message [ 524.820126] Lustre: Skipped 3 previous similar messages [ 530.105861] Lustre: server umount lustre-OST0000 complete [ 535.008107] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 535.011762] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 535.024984] Lustre: Skipped 1 previous similar message [ 535.046276] Lustre: Skipped 3 previous similar messages [ 540.796750] Lustre: server umount lustre-MDT0000 complete [ 544.796550] LustreError: 15067:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788441099 with bad export cookie 6997162701422351050 [ 544.803957] LustreError: MGC192.168.202.148@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 544.807899] LustreError: 15067:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 545.020796] Lustre: server umount lustre-MDT0001 complete [ 550.940433] Lustre: DEBUG MARKER: == conf-sanity test 76a: set permanent params with lctl across mounts ========================================================== 09:11:44 (1788441104) [ 557.359087] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 558.169746] LustreError: 19310:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 558.197469] LustreError: 19310:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 558.281031] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 562.633191] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 563.685901] LustreError: 19311:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 573.802605] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 573.920763] LustreError: 19310:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 573.931235] LustreError: 19310:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 580.763465] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 593.286168] Lustre: DEBUG MARKER: oleg248-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 600.902036] Lustre: DEBUG MARKER: oleg248-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 608.415443] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 616.796359] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 624.994527] Lustre: DEBUG MARKER: oleg248-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 633.974832] Lustre: Modifying parameter general.osc.*.max_dirty_mb=452 in log params [ 644.070257] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 644.079088] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 644.087405] Lustre: Skipped 2 previous similar messages [ 644.107161] Lustre: Skipped 4 previous similar messages [ 647.978687] Lustre: server umount lustre-MDT0000 complete [ 649.189671] LustreError: 20039:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 649.216142] LustreError: 20039:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 651.773400] LustreError: 20046:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788441206 with bad export cookie 6997162701422351939 [ 651.775157] LustreError: MGC192.168.202.148@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 651.788503] LustreError: 20046:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 652.358774] Lustre: server umount lustre-MDT0001 complete [ 666.571291] Lustre: server umount lustre-OST0000 complete [ 689.742526] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing load_modules_local [ 700.727569] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 701.321590] LustreError: 23697:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 701.438255] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 701.441309] Lustre: Skipped 2 previous similar messages [ 706.099477] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 716.547781] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 716.772297] Lustre: lustre-MDT0001: Not available for connect from 0@lo (not set up) [ 722.353783] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 725.822913] Lustre: Setting parameter general.debug_raw_pointers=Y in log params [ 733.822300] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 740.787380] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 749.159674] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 749.432124] Lustre: lustre-OST0001: new disk, initializing [ 749.443983] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 749.450745] Lustre: 26012:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 751.249103] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 751.258457] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 751.494537] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 756.904562] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 767.090062] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 770.784951] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 783.843761] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 783.855411] LustreError: Skipped 1 previous similar message [ 783.857295] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 783.868141] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 790.171969] Lustre: server umount lustre-MDT0000 complete [ 792.547659] LustreError: 23698:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 792.561843] LustreError: 23698:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 5 previous similar messages [ 794.559363] LustreError: 23679:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788441349 with bad export cookie 6997162701422352786 [ 794.570352] LustreError: MGC192.168.202.148@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 794.580070] LustreError: 23679:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 795.291673] Lustre: server umount lustre-MDT0001 complete [ 809.142131] Lustre: server umount lustre-OST0000 complete [ 813.023167] Lustre: 3641:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1788441352/real 1788441352] req@ffff9337022bdc00 x1875315967384064/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1788441368 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 813.046465] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 813.063284] Lustre: Skipped 3 previous similar messages [ 813.518573] Lustre: server umount lustre-OST0001 complete [ 834.478502] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing load_modules_local [ 846.585456] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 847.099331] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 847.104525] Lustre: Skipped 3 previous similar messages [ 851.995114] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 860.959203] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 866.426452] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 870.045337] Lustre: 30180:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 877.567901] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 884.793220] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 894.084343] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 901.873709] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 910.343831] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 914.024511] Lustre: 32061:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 922.600176] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 922.619921] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 922.645209] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 922.653586] Lustre: Skipped 4 previous similar messages [ 926.895254] Lustre: server umount lustre-MDT0000 complete [ 930.278556] LustreError: 29031:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 930.306500] LustreError: 29031:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 10 previous similar messages [ 930.979395] LustreError: 29012:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788441486 with bad export cookie 6997162701422354081 [ 930.989320] LustreError: MGC192.168.202.148@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 930.992395] LustreError: 29012:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 931.626238] Lustre: server umount lustre-MDT0001 complete [ 946.351306] Lustre: server umount lustre-OST0000 complete [ 960.283916] Lustre: server umount lustre-OST0001 complete [ 969.878814] Lustre: DEBUG MARKER: == conf-sanity test 76b: verify params log setup correctly ========================================================== 09:18:44 (1788441524) [ 1001.012887] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing load_modules_local [ 1014.294833] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1021.117848] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 1031.443099] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1036.204789] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 1039.400465] Lustre: 36018:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1046.559193] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1054.159344] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 1063.885567] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1071.709633] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 1080.354717] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1083.789918] Lustre: 37897:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 1090.024534] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1090.039327] Lustre: Skipped 5 previous similar messages [ 1094.939484] Lustre: server umount lustre-MDT0000 complete [ 1099.198932] LustreError: 34852:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788441654 with bad export cookie 6997162701422355187 [ 1099.200255] LustreError: MGC192.168.202.148@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1099.209935] LustreError: 34852:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 1099.610279] Lustre: server umount lustre-MDT0001 complete [ 1114.847866] Lustre: server umount lustre-OST0000 complete [ 1116.127251] Lustre: 3643:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1788441655/real 1788441655] req@ffff93383c3bed80 x1875315967487232/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1788441671 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1119.426743] Lustre: server umount lustre-OST0001 complete [ 1129.734464] Lustre: DEBUG MARKER: == conf-sanity test 76c: verify changelog_mask is applied with lctl set_param -P ========================================================== 09:21:23 (1788441683) [ 1149.238459] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing load_modules_local [ 1161.689386] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1162.297738] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1162.305601] Lustre: Skipped 7 previous similar messages [ 1167.761374] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 1176.932546] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1182.612415] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 1185.785693] Lustre: 41187:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1193.933097] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1201.849171] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 1203.814593] LustreError: 41541:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1203.837294] LustreError: 41541:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 14 previous similar messages [ 1211.203963] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1219.144943] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 1227.586769] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1231.584921] Lustre: 43065:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 1232.495899] Lustre: Setting parameter general.mdd.*.changelog_mask=-CLOSE in log params [ 1232.504131] Lustre: Skipped 1 previous similar message [ 1237.475510] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1237.481181] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1237.500367] Lustre: Skipped 5 previous similar messages [ 1237.518719] Lustre: Skipped 11 previous similar messages [ 1238.337875] Lustre: server umount lustre-MDT0000 complete [ 1242.613143] LustreError: 40020:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788441797 with bad export cookie 6997162701422356293 [ 1242.615275] LustreError: MGC192.168.202.148@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1242.623607] LustreError: 40020:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 1249.144148] Lustre: server umount lustre-MDT0001 complete [ 1259.184873] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1272.750259] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 1276.902692] Lustre: lustre-MDT0000-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 1283.072070] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1287.552986] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 1287.560984] Lustre: Skipped 1 previous similar message [ 1288.241303] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 1297.142194] Lustre: DEBUG MARKER: oleg248-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1299.035275] Lustre: DEBUG MARKER: mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec [ 1305.450781] Lustre: DEBUG MARKER: oleg248-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 1306.960868] Lustre: DEBUG MARKER: mdc.lustre-MDT0001-mdc-*.mds_server_uuid in FULL state after 0 sec [ 1318.959765] Lustre: server umount lustre-MDT0000 complete [ 1324.722170] LustreError: 40020:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788441879 with bad export cookie 6997162701422357385 [ 1324.739089] LustreError: MGC192.168.202.148@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1325.293073] Lustre: server umount lustre-MDT0001 complete [ 1340.310397] Lustre: server umount lustre-OST0000 complete [ 1353.959379] Lustre: server umount lustre-OST0001 complete [ 1363.931411] Lustre: DEBUG MARKER: == conf-sanity test 76d: verify llite.*.xattr_cache can be set by 'lctl set_param -P' correctly ========================================================== 09:25:18 (1788441918) [ 1381.371629] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing load_modules_local [ 1393.059530] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1397.720874] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 1406.765673] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1413.057579] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 1415.957484] Lustre: 48830:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1424.570608] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1432.120342] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 1441.031656] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1448.295859] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 1455.666979] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1460.228568] Lustre: Setting parameter general.llite.*.xattr_cache=0 in log params [ 1474.022581] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1476.309707] Lustre: server umount lustre-MDT0000 complete [ 1480.165048] LustreError: 47658:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788442035 with bad export cookie 6997162701422358211 [ 1480.167362] LustreError: MGC192.168.202.148@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1480.188380] LustreError: 47658:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 7 previous similar messages [ 1480.633941] Lustre: server umount lustre-MDT0001 complete [ 1494.728674] Lustre: server umount lustre-OST0000 complete [ 1498.080661] Lustre: 3642:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1788442037/real 1788442037] req@ffff933810a1f480 x1875315967633792/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1788442053 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1498.118044] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1498.139275] Lustre: Skipped 10 previous similar messages [ 1499.001486] Lustre: server umount lustre-OST0001 complete [ 1509.777895] Lustre: DEBUG MARKER: == conf-sanity test 77: comma-separated MGS NIDs and failover node NIDs ========================================================== 09:27:43 (1788442063) [ 1512.225653] Lustre: DEBUG MARKER: SKIP: conf-sanity test_77 mixed loopback and real device not working [ 1514.396760] Lustre: DEBUG MARKER: == conf-sanity test 78: run resize2fs on MDT and OST filesystems ========================================================== 09:27:48 (1788442068) [ 1524.751769] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_hostid [ 1534.504719] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing load_modules_local [ 1559.140520] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1559.406458] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1559.480686] Lustre: lustre-MDT0000: new disk, initializing [ 1559.590374] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1564.005622] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 1573.082292] Lustre: DEBUG MARKER: oleg248-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1581.460825] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1581.854467] Lustre: lustre-OST0000: new disk, initializing [ 1581.861200] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1581.865623] Lustre: Skipped 1 previous similar message [ 1581.870615] Lustre: 54902:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1583.571694] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 1583.588696] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 1583.634570] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 1589.939279] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 1600.449969] Lustre: DEBUG MARKER: oleg248-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1602.903978] Lustre: DEBUG MARKER: create test files [ 1634.791862] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1639.593302] Lustre: server umount lustre-OST0000 complete [ 1643.671096] Lustre: server umount lustre-MDT0000 complete [ 1656.536896] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing unload_modules_local [ 1659.906381] Key type lgssc unregistered [ 1660.256760] LNet: 56540:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1660.264717] LNetError: 56540:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1660.284090] LNet: Removed LNI 192.168.202.148@tcp [ 1661.477138] Key type .llcrypt unregistered [ 1661.481314] Key type ._llcrypt unregistered [ 1693.862923] Key type ._llcrypt registered [ 1693.864934] Key type .llcrypt registered [ 1693.979553] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing load_modules_local [ 1695.704511] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1695.747550] alg: No test for adler32 (adler32-zlib) [ 1696.891604] Lustre: Lustre: Build Version: 2.17.56_51_g2aa7cea [ 1697.179309] LNet: Added LNI 192.168.202.148@tcp [8/256/0/180] [ 1698.903734] Key type lgssc registered [ 1700.207396] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1709.026429] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1709.065225] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1710.533109] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1715.125225] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 1722.705991] Lustre: DEBUG MARKER: oleg248-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1729.709584] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1729.938272] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1735.121539] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:98 to 0x240000400:129) [ 1735.289258] Lustre: 59275:0:(osd_internal.h:1461:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 511 < left 544, rollback = 9 [ 1735.314882] Lustre: 59275:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 32/128/4, destroy: 0/0/0 [ 1735.327492] Lustre: 59275:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 1735.343924] Lustre: 59275:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/18/1, punch: 0/0/0, quota 1/3/0 [ 1735.350978] Lustre: 59275:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 1735.360620] Lustre: 59275:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 1737.385526] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 1745.346954] Lustre: DEBUG MARKER: oleg248-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1750.173932] Lustre: DEBUG MARKER: check files after expanding the MDT and OST filesystems [ 1755.255916] Lustre: DEBUG MARKER: create more files after expanding the MDT and OST filesystems [ 1765.793403] Lustre: server umount lustre-OST0000 complete [ 1779.854978] Lustre: server umount lustre-MDT0000 complete [ 1792.250340] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing unload_modules_local [ 1795.163537] Key type lgssc unregistered [ 1795.487994] LNet: 60890:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1795.504745] LNetError: 60890:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1795.533406] LNet: Removed LNI 192.168.202.148@tcp [ 1796.508403] Key type .llcrypt unregistered [ 1796.511622] Key type ._llcrypt unregistered [ 1831.372522] Key type ._llcrypt registered [ 1831.375308] Key type .llcrypt registered [ 1831.584131] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing load_modules_local [ 1832.628077] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1832.747193] alg: No test for adler32 (adler32-zlib) [ 1833.949840] Lustre: Lustre: Build Version: 2.17.56_51_g2aa7cea [ 1834.199595] LNet: Added LNI 192.168.202.148@tcp [8/256/0/180] [ 1835.929604] Key type lgssc registered [ 1837.532893] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1846.479096] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1846.506053] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1848.007314] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1853.344566] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 1861.136829] Lustre: DEBUG MARKER: oleg248-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1867.820705] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1868.253779] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1873.410384] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:140 to 0x240000400:161) [ 1873.525283] Lustre: 63728:0:(osd_internal.h:1461:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 509 < left 544, rollback = 9 [ 1873.548224] Lustre: 63728:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 32/128/6, destroy: 0/0/0 [ 1873.560846] Lustre: 63728:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 1873.576044] Lustre: 63728:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/18/1, punch: 0/0/0, quota 1/3/0 [ 1873.595910] Lustre: 63728:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 1873.607165] Lustre: 63728:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 1875.853348] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 1884.850657] Lustre: DEBUG MARKER: oleg248-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1890.405355] Lustre: DEBUG MARKER: check files after shrinking the MDT and OST filesystems [ 1904.101395] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1904.107342] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1904.114737] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1906.973151] Lustre: server umount lustre-OST0000 complete [ 1911.408150] Lustre: server umount lustre-MDT0000 complete [ 1923.739787] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing unload_modules_local [ 1926.813509] Key type lgssc unregistered [ 1927.316867] LNet: 65288:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1927.330103] LNetError: 65288:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1927.345832] LNet: Removed LNI 192.168.202.148@tcp [ 1928.387400] Key type .llcrypt unregistered [ 1928.394112] Key type ._llcrypt unregistered [ 1947.330892] Key type ._llcrypt registered [ 1947.335673] Key type .llcrypt registered [ 1947.468588] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_hostid [ 1962.958298] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing load_modules_local [ 1963.996556] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1964.055084] alg: No test for adler32 (adler32-zlib) [ 1965.231395] Lustre: Lustre: Build Version: 2.17.56_51_g2aa7cea [ 1965.506709] LNet: Added LNI 192.168.202.148@tcp [8/256/0/180] [ 1967.191162] Key type lgssc registered [ 1968.274541] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2002.951778] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 2002.986736] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2004.283326] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2004.330284] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2004.467380] Lustre: lustre-MDT0000: new disk, initializing [ 2004.592509] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2004.622481] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2009.195924] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 2019.345161] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2019.454656] Lustre: 68442:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2019.497635] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 2019.500046] Lustre: Skipped 1 previous similar message [ 2019.558708] Lustre: lustre-MDT0001: new disk, initializing [ 2019.655302] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2019.714211] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 2019.726087] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2024.952567] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 2035.229552] Lustre: DEBUG MARKER: oleg248-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2040.494192] Lustre: DEBUG MARKER: oleg248-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2047.251105] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2047.567428] Lustre: lustre-OST0000: new disk, initializing [ 2047.571745] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 2047.587601] Lustre: 70411:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2047.752963] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2054.160687] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 2056.772937] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 2056.783667] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2056.969237] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2063.695717] Lustre: DEBUG MARKER: oleg248-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2069.251490] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2069.533943] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2074.922119] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2075.148889] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2077.679757] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2077.700373] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2077.723982] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2077.733749] Lustre: Skipped 1 previous similar message [ 2082.798461] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2082.816954] Lustre: Skipped 1 previous similar message [ 2083.697725] Lustre: server umount lustre-OST0000 complete [ 2091.491897] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2091.504109] LustreError: Skipped 1 previous similar message [ 2091.508538] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2091.522262] Lustre: Skipped 1 previous similar message [ 2091.532752] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2093.027660] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2094.318266] Lustre: server umount lustre-MDT0000 complete [ 2098.428517] LustreError: 68434:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788442653 with bad export cookie 12540305632762714303 [ 2098.445947] LustreError: MGC192.168.202.148@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2098.453453] LustreError: 68434:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 2099.073518] Lustre: server umount lustre-MDT0001 complete [ 2108.622297] Lustre: DEBUG MARKER: == conf-sanity test 79: format MDT/OST without mgs option (should return errors) ========================================================== 09:37:42 (1788442662) [ 2120.062273] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing load_modules_local [ 2162.674918] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_hostid [ 2173.445945] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing load_modules_local [ 2212.976196] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2213.228222] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2213.269900] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2213.353403] Lustre: lustre-MDT0000: new disk, initializing [ 2213.444264] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2213.463700] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2218.153693] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 2228.825636] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2229.006578] Lustre: 76431:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2229.090201] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 2229.093896] Lustre: Skipped 1 previous similar message [ 2229.196940] Lustre: lustre-MDT0001: new disk, initializing [ 2229.315975] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2229.348803] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 2229.355630] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2234.005927] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 2243.314413] Lustre: DEBUG MARKER: oleg248-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2248.728426] Lustre: DEBUG MARKER: oleg248-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2255.657940] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2255.920395] Lustre: lustre-OST0000: new disk, initializing [ 2255.924396] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 2255.929795] Lustre: 78402:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2256.032567] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2262.488543] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 2265.689972] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 2265.713704] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2265.849392] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2271.971211] Lustre: DEBUG MARKER: oleg248-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2276.642358] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2276.875411] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2281.968952] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2282.246946] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2286.568913] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2286.578776] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2286.579954] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2286.585690] Lustre: Skipped 1 previous similar message [ 2286.597021] Lustre: Skipped 2 previous similar messages [ 2290.667640] Lustre: server umount lustre-OST0000 complete [ 2295.778655] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2295.788301] LustreError: Skipped 1 previous similar message [ 2295.790340] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2295.808786] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2295.811426] Lustre: Skipped 1 previous similar message [ 2306.016119] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2306.026184] Lustre: Skipped 5 previous similar messages [ 2309.087889] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2309.467157] Lustre: server umount lustre-MDT0000 complete [ 2311.141791] LustreError: 76444:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2311.172995] LustreError: 76444:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 4 previous similar messages [ 2313.357338] LustreError: 76425:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788442868 with bad export cookie 12540305632762715185 [ 2313.361418] LustreError: MGC192.168.202.148@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2313.369372] LustreError: 76425:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2313.673693] Lustre: server umount lustre-MDT0001 complete [ 2323.543894] Lustre: DEBUG MARKER: == conf-sanity test 80: mgc import reconnect race ======== 09:41:17 (1788442877) [ 2330.807066] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2331.342501] LustreError: 80815:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2331.398274] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2335.597442] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 2336.748164] LustreError: 80816:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2340.834656] LustreError: 80815:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2344.144148] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2349.745515] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 2357.713436] Lustre: DEBUG MARKER: oleg248-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2364.424288] Lustre: DEBUG MARKER: oleg248-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2371.612152] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2372.193553] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2372.202035] Lustre: Skipped 1 previous similar message [ 2379.156976] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 2386.459166] Lustre: DEBUG MARKER: oleg248-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2390.604980] Lustre: 83487:0:(genops.c:1795:obd_export_evict_by_uuid()) MGS: evicting 0402b621-08f5-47d0-81e3-2f0e1126e1e1 at adminstrative request [ 2395.129307] LustreError: MGC192.168.202.148@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2395.157587] Lustre: *** cfs_fail_loc=906, val=2147483648*** [ 2405.382150] Lustre: MGS: Client 0402b621-08f5-47d0-81e3-2f0e1126e1e1 (at 0@lo) reconnecting [ 2405.392195] Lustre: Evicted from MGS (at 0@lo) after server handle changed from 0xae081d8817a6df9c to 0xae081d8817a6e243 [ 2405.404058] Lustre: MGC192.168.202.148@tcp: Connection restored to 0@lo (at 0@lo) [ 2428.402249] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2428.635240] Lustre: lustre-OST0001: new disk, initializing [ 2428.639660] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 2428.648508] Lustre: 83881:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2435.058268] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 2436.250934] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 2436.279705] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 2436.406833] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 2444.170497] Lustre: DEBUG MARKER: oleg248-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2446.822904] Lustre: lustre-OST0001-osc-MDT0000: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2446.839682] Lustre: Skipped 2 previous similar messages [ 2446.844215] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 2446.844240] LustreError: lustre-OST0001-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2446.848959] Lustre: Skipped 1 previous similar message [ 2461.151864] Lustre: lustre-OST0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2461.264545] Lustre: server umount lustre-OST0001 complete [ 2462.200407] LustreError: 82637:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2462.235074] LustreError: 82637:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 2467.313277] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2467.328829] Lustre: Skipped 1 previous similar message [ 2472.439130] LustreError: 82638:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2472.453184] LustreError: 82638:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [ 2473.660703] Lustre: server umount lustre-OST0000 complete [ 2482.661290] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2482.662761] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2482.687881] Lustre: Skipped 9 previous similar messages [ 2485.644753] Lustre: server umount lustre-MDT0000 complete [ 2489.147243] LustreError: 80795:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788443044 with bad export cookie 12540305632762716739 [ 2489.159376] LustreError: MGC192.168.202.148@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2489.168630] LustreError: 80795:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2489.618447] Lustre: server umount lustre-MDT0001 complete [ 2498.706406] Lustre: DEBUG MARKER: == conf-sanity test 81: sparse OST indexing ============== 09:44:13 (1788443053) [ 2500.596633] Lustre: DEBUG MARKER: SKIP: conf-sanity test_81 needs >= 3 OSTs [ 2502.753627] Lustre: DEBUG MARKER: == conf-sanity test 82a: specify OSTs for file (succeed) or directory (succeed) ========================================================== 09:44:16 (1788443056) [ 2504.339641] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82a needs >= 3 OSTs [ 2506.150679] Lustre: DEBUG MARKER: == conf-sanity test 82b: specify OSTs for file with --pool and --ost-list options ========================================================== 09:44:20 (1788443060) [ 2507.642380] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82b needs >= 4 OSTs [ 2509.545854] Lustre: DEBUG MARKER: == conf-sanity test 82c: specify sparse OSTs for setstripe (should not crash) ========================================================== 09:44:23 (1788443063) [ 2510.968120] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82c needs >= 4 OSTs [ 2512.491488] Lustre: DEBUG MARKER: == conf-sanity test 83: ENOSPACE on OST doesn't cause message VFS: Busy inodes after unmount ... ========================================================== 09:44:27 (1788443067) [ 2514.068751] Lustre: DEBUG MARKER: mount the OST /dev/mapper/ost1_flakey as a ldiskfs filesystem [ 2520.213191] loop: module loaded [ 2520.238950] loop0: detected capacity change from 0 to 4294967296 [ 2520.283328] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 2521.945927] Lustre: DEBUG MARKER: run llverfs in partial mode on the OST ldiskfs /mnt/lustre-ost1 [ 2525.557953] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing run_llverfs /mnt/lustre-ost1 -vpl no [ 2531.474242] Lustre: DEBUG MARKER: unmount the OST /dev/mapper/ost1_flakey [ 2559.033426] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2559.092300] LustreError: 88008:0:(obd_config.c:845:class_setup()) setup lustre-OST0000-osd failed (-28) [ 2559.115178] LustreError: 88008:0:(obd_mount.c:259:lustre_start_simple()) lustre-OST0000-osd setup error -28 [ 2559.131520] LustreError: 88008:0:(tgt_mount.c:2579:server_fill_super()) Unable to start osd on /dev/mapper/ost1_flakey: -28 [ 2559.139676] LustreError: 88008:0:(super25.c:178:lustre_fill_super()) llite: Unable to mount : rc = -28 [ 2580.075488] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_hostid [ 2592.632685] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing load_modules_local [ 2632.017539] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2632.428149] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2632.490874] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2632.591221] Lustre: lustre-MDT0000: new disk, initializing [ 2632.694040] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2632.699647] Lustre: Skipped 1 previous similar message [ 2632.737691] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2637.905677] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 2651.742477] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2651.991726] Lustre: 90875:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2652.426777] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2656.575869] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 2667.074540] Lustre: DEBUG MARKER: oleg248-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2674.045183] Lustre: DEBUG MARKER: oleg248-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2681.483157] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2681.865445] Lustre: 92844:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2688.155405] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2688.324795] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2688.927336] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 2699.554787] Lustre: DEBUG MARKER: oleg248-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2704.283910] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2704.463644] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2709.577947] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2710.067280] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2714.080880] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2714.081785] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2714.093731] LustreError: Skipped 1 previous similar message [ 2714.094607] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2714.131304] Lustre: Skipped 3 previous similar messages [ 2718.709646] Lustre: server umount lustre-OST0000 complete [ 2728.211128] Lustre: server umount lustre-MDT0000 complete [ 2729.453344] LustreError: 91974:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2729.483248] LustreError: 91974:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [ 2732.153395] LustreError: 90867:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788443287 with bad export cookie 12540305632762717348 [ 2732.161794] LustreError: MGC192.168.202.148@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2732.167964] LustreError: 90867:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2732.533651] Lustre: server umount lustre-MDT0001 complete [ 2740.244354] Lustre: DEBUG MARKER: == conf-sanity test 84: check recovery_hard_time ========= 09:48:14 (1788443294) [ 2748.938078] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2756.189688] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 2764.270297] LustreError: 95254:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2764.290700] LustreError: 95254:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [ 2765.912705] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2766.516802] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-60 [ 2766.533616] Lustre: Skipped 3 previous similar messages [ 2771.581984] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 2779.291171] Lustre: DEBUG MARKER: oleg248-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2785.706159] Lustre: DEBUG MARKER: oleg248-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2792.615644] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2799.919785] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 2807.004672] Lustre: DEBUG MARKER: oleg248-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2812.160190] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2812.393910] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2817.675463] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2817.854737] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2825.322304] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2825.570429] Lustre: lustre-OST0001: new disk, initializing [ 2825.576957] Lustre: Skipped 2 previous similar messages [ 2825.584142] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 2825.591594] Lustre: Skipped 3 previous similar messages [ 2825.606338] Lustre: 98625:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2832.910973] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 2835.117086] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 2835.136428] Lustre: Skipped 2 previous similar messages [ 2835.164460] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 2835.325151] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 2843.936520] Lustre: DEBUG MARKER: oleg248-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2849.562739] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid 50 [ 2849.808977] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2855.394591] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [ 2855.624902] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2876.369172] Lustre: DEBUG MARKER: mds1 REPLAY BARRIER on lustre-MDT0000 [ 2901.798880] Lustre: Failing over lustre-MDT0000 [ 2901.991632] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2902.008334] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2902.010649] Lustre: Skipped 4 previous similar messages [ 2902.029676] Lustre: Skipped 7 previous similar messages [ 2902.294086] Lustre: server umount lustre-MDT0000 complete [ 2904.112113] LustreError: 98880:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 192.168.202.48@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2904.133189] LustreError: 98880:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 2923.487202] Lustre: 66667:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1788443462/real 1788443462] req@00000000c16547a1 x1875317904131840/t0(0) o400->MGC192.168.202.148@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1788443478 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 2923.519712] LustreError: MGC192.168.202.148@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2928.187832] LDISKFS-fs (dm-0): 4 truncates cleaned up [ 2928.191106] LDISKFS-fs (dm-0): recovery complete [ 2928.198237] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2933.733104] LustreError: 66664:0:(client.c:1391:ptlrpc_import_delay_req()) @@@ invalidate in flight req@0000000039b8f3d2 x1875317904139648/t0(0) o250->MGC192.168.202.148@tcp@0@lo:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl Rpc:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 2934.119263] Lustre: lustre-MDT0000: in recovery but waiting for the first client to connect [ 2934.790984] Lustre: lustre-MDT0000: Will be in recovery for at least 1:00, or until 3 clients reconnect [ 2934.791259] Lustre: 95250:0:(ldlm_lib.c:2085:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 2938.566612] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 2939.365900] Lustre: 95255:0:(ldlm_lib.c:2085:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 2939.384230] Lustre: 95255:0:(ldlm_lib.c:2085:extend_recovery_timer()) Skipped 1 previous similar message [ 2939.420496] Lustre: lustre-MDT0000-lwp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 2939.509977] LustreError: 101477:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2939.832052] LustreError: 101477:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2940.154861] LustreError: 101477:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2940.166308] LustreError: 101477:0:(ldlm_lib.c:2698:replay_request_or_update()) Skipped 1 previous similar message [ 2940.495275] LustreError: 101477:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2940.505287] LustreError: 101477:0:(ldlm_lib.c:2698:replay_request_or_update()) Skipped 1 previous similar message [ 2940.510515] Lustre: 101477:0:(ldlm_lib.c:2085:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2940.515647] Lustre: 101477:0:(ldlm_lib.c:2085:extend_recovery_timer()) Skipped 7 previous similar messages [ 2941.470358] LustreError: 101477:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2941.477308] LustreError: 101477:0:(ldlm_lib.c:2698:replay_request_or_update()) Skipped 3 previous similar messages [ 2941.825571] LustreError: 101477:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2941.841074] LustreError: 101477:0:(ldlm_lib.c:2698:replay_request_or_update()) Skipped 3 previous similar messages [ 2942.801684] Lustre: 101477:0:(ldlm_lib.c:2085:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2942.815607] Lustre: 101477:0:(ldlm_lib.c:2085:extend_recovery_timer()) Skipped 6 previous similar messages [ 2943.771715] LustreError: 101477:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2943.788158] LustreError: 101477:0:(ldlm_lib.c:2698:replay_request_or_update()) Skipped 6 previous similar messages [ 2944.113472] LustreError: 101477:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2944.115786] LustreError: 101477:0:(ldlm_lib.c:2698:replay_request_or_update()) Skipped 6 previous similar messages [ 2947.000987] Lustre: 101477:0:(ldlm_lib.c:2085:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2947.009235] Lustre: 101477:0:(ldlm_lib.c:2085:extend_recovery_timer()) Skipped 12 previous similar messages [ 2947.985375] LustreError: 101477:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2947.989638] LustreError: 101477:0:(ldlm_lib.c:2698:replay_request_or_update()) Skipped 12 previous similar messages [ 2948.305642] LustreError: 101477:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2948.308532] LustreError: 101477:0:(ldlm_lib.c:2698:replay_request_or_update()) Skipped 12 previous similar messages [ 2955.026227] Lustre: 101477:0:(ldlm_lib.c:2085:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2955.048509] Lustre: 101477:0:(ldlm_lib.c:2085:extend_recovery_timer()) Skipped 24 previous similar messages [ 2956.024478] LustreError: 101477:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2956.051033] LustreError: 101477:0:(ldlm_lib.c:2698:replay_request_or_update()) Skipped 24 previous similar messages [ 2956.399322] LustreError: 101477:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2956.424024] LustreError: 101477:0:(ldlm_lib.c:2698:replay_request_or_update()) Skipped 24 previous similar messages [ 2971.107275] Lustre: 101477:0:(ldlm_lib.c:2085:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2971.111652] Lustre: 101477:0:(ldlm_lib.c:2085:extend_recovery_timer()) Skipped 49 previous similar messages [ 2972.058681] LustreError: 101477:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2972.068037] LustreError: 101477:0:(ldlm_lib.c:2698:replay_request_or_update()) Skipped 49 previous similar messages [ 2972.703152] LustreError: 101477:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2972.707332] LustreError: 101477:0:(ldlm_lib.c:2698:replay_request_or_update()) Skipped 50 previous similar messages [ 2994.199217] Lustre: 101477:0:(ldlm_lib.c:2394:target_recovery_overseer()) lustre-MDT0000 recovery is aborted by hard timeout [ 2994.209394] Lustre: 101477:0:(ldlm_lib.c:2404:target_recovery_overseer()) recovery is aborted, evict exports in recovery [ 2994.216684] Lustre: 101477:0:(genops.c:1622:class_disconnect_stale_exports()) lustre-MDT0000: disconnect stale client 3b91e471-9bca-43f6-ad9a-9eb4c3fe8d8d@192.168.202.48@tcp [ 2994.222106] Lustre: lustre-MDT0000: disconnecting 1 stale clients [ 2994.225253] Lustre: 101477:0:(ldlm_lib.c:1914:abort_req_replay_queue()) @@@ aborted: req@00000000cbdecf62 x1875317889240064/t0(8589934768) o35->3b91e471-9bca-43f6-ad9a-9eb4c3fe8d8d@192.168.202.48@tcp:315/0 lens 392/0 e 0 to 0 dl 1788443560 ref 1 fl Complete:/604/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:0 [ 2994.241084] Lustre: lustre-MDT0000-osd: cancel update llog [0x200000400:0x1:0x0] [ 2994.257136] Lustre: lustre-MDT0000: Denying connection for new client 3b91e471-9bca-43f6-ad9a-9eb4c3fe8d8d (at 192.168.202.48@tcp), waiting for 3 known clients (2 recovered, 0 in progress, and 1 evicted) to recover in 0:00 [ 2994.276908] Lustre: lustre-MDT0001-osp-MDT0000: cancel update llog [0x240000401:0x1:0x0] [ 2994.321592] Lustre: lustre-MDT0000-osp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 2994.325817] Lustre: Skipped 2 previous similar messages [ 2994.335408] Lustre: lustre-MDT0000: Recovery over after 1:00, of 3 clients 2 recovered and 1 was evicted. [ 2994.391929] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:77 to 0x280000401:577) [ 2994.393674] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:45 to 0x2c0000401:545) [ 3013.548478] Lustre: server umount lustre-OST0000 complete [ 3019.745144] LustreError: lustre-OST0001-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3019.760392] LustreError: Skipped 1 previous similar message [ 3024.644537] Lustre: server umount lustre-OST0001 complete [ 3030.497494] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3030.508130] Lustre: Skipped 5 previous similar messages [ 3036.649115] LustreError: 95254:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3036.668535] LustreError: 95254:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 43 previous similar messages [ 3036.752444] Lustre: server umount lustre-MDT0000 complete [ 3040.737821] LustreError: 95234:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788443595 with bad export cookie 12540305632762768574 [ 3040.739912] LustreError: MGC192.168.202.148@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3040.747217] LustreError: 95234:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 3041.030835] Lustre: server umount lustre-MDT0001 complete [ 3049.992152] Lustre: DEBUG MARKER: == conf-sanity test 85: osd_ost init: fail ea_fid_set ==== 09:53:24 (1788443604) [ 3057.361161] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3057.365953] Lustre: *** cfs_fail_loc=197, val=0*** [ 3072.799559] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 3077.919530] LustreError: 103454:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.202.148@tcp: failed processing log, type 4: rc = -110 [ 3083.059231] Lustre: *** cfs_fail_loc=197, val=0*** [ 3083.062683] Lustre: Skipped 65 previous similar messages [ 3103.583802] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3103.588130] Lustre: Skipped 3 previous similar messages [ 3110.258892] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 3117.377688] Lustre: DEBUG MARKER: oleg248-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3119.393348] Lustre: server umount lustre-OST0000 complete [ 3126.981326] Lustre: DEBUG MARKER: == conf-sanity test 86: Replacing mkfs.lustre -G option == 09:54:41 (1788443681) [ 3147.325264] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3152.172601] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 3161.841436] Lustre: DEBUG MARKER: oleg248-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3166.697347] LustreError: 105110:0:(osp_object.c:618:osp_attr_get()) lustre-MDT0001-osp-MDT0000: osp_attr_get update error [0x200000009:0x1:0x0]: rc = -5 [ 3166.717931] LustreError: 105110:0:(lod_sub_object.c:917:lod_sub_prep_llog()) lustre-MDT0000-mdtlov: can't get id from catalogs: rc = -5 [ 3166.727887] LustreError: 105110:0:(lod_dev.c:511:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 19, retries 0, failed: rc = -5 [ 3167.016294] Lustre: server umount lustre-MDT0000 complete [ 3210.949337] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_hostid [ 3222.109983] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing load_modules_local [ 3259.116358] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3259.314237] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3259.333813] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3259.414950] Lustre: lustre-MDT0000: new disk, initializing [ 3259.495471] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3264.793971] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 3278.152937] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3278.358453] Lustre: 109286:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 3278.701905] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3283.771262] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 3296.096210] Lustre: DEBUG MARKER: oleg248-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3304.541262] Lustre: DEBUG MARKER: oleg248-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3311.985659] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3312.382131] Lustre: 111255:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3318.467086] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3319.126045] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 3328.014392] Lustre: DEBUG MARKER: oleg248-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3332.645164] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 3332.830333] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 3337.078626] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 3337.336932] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 3344.352073] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3344.355441] LustreError: Skipped 2 previous similar messages [ 3344.356109] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3344.365464] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3344.365472] Lustre: Skipped 9 previous similar messages [ 3344.402142] Lustre: Skipped 2 previous similar messages [ 3345.532589] Lustre: server umount lustre-OST0000 complete [ 3355.405276] Lustre: server umount lustre-MDT0000 complete [ 3359.386710] LustreError: 109276:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788443914 with bad export cookie 12540305632762779144 [ 3359.403883] LustreError: MGC192.168.202.148@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3359.602546] Lustre: server umount lustre-MDT0001 complete [ 3367.996346] Lustre: DEBUG MARKER: == conf-sanity test 87: check if MDT inode can hold EAs with N stripes properly ========================================================== 09:58:42 (1788443922) [ 3382.344975] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing unload_modules_local [ 3387.912767] Key type lgssc unregistered [ 3388.579842] LNet: 113744:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3388.593545] LNetError: 113744:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3389.672571] LNet: Removed LNI 192.168.202.148@tcp [ 3391.772735] Key type .llcrypt unregistered [ 3391.774546] Key type ._llcrypt unregistered [ 3411.469808] Key type ._llcrypt registered [ 3411.471657] Key type .llcrypt registered [ 3411.572961] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_hostid [ 3428.612359] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing load_modules_local [ 3430.220413] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3430.283231] alg: No test for adler32 (adler32-zlib) [ 3431.565836] Lustre: Lustre: Build Version: 2.17.56_51_g2aa7cea [ 3431.742650] LNet: Added LNI 192.168.202.148@tcp [8/256/0/180] [ 3433.375444] Key type lgssc registered [ 3434.737678] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3476.091766] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3476.167162] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3477.432827] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3477.460140] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3477.519772] Lustre: lustre-MDT0000: new disk, initializing [ 3477.569896] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3477.585235] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3482.824821] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 3493.708491] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3494.042962] Lustre: lustre-OST0000: new disk, initializing [ 3494.046782] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3494.050786] Lustre: Skipped 1 previous similar message [ 3494.058934] Lustre: 117920:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3494.159863] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3496.200980] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 3496.218749] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 3496.274386] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 3501.888790] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 3516.695802] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3517.087824] Lustre: lustre-OST0001: new disk, initializing [ 3517.090563] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 3517.109712] Lustre: 118796:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3517.244712] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 3524.879376] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:1:ost [ 3524.888896] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:1:ost] [ 3524.933418] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x280000400 [ 3525.074380] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 3535.341337] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3535.356197] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3535.367209] Lustre: Skipped 1 previous similar message [ 3540.448395] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3540.463902] Lustre: Skipped 2 previous similar messages [ 3541.628123] Lustre: server umount lustre-MDT0000 complete [ 3549.333429] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: (null) [ 3554.367927] LustreError: 117106:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788444109 with bad export cookie 12918797705063485973 [ 3554.380959] LustreError: MGC192.168.202.148@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3554.388193] LustreError: 117106:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 3554.515868] Lustre: server umount lustre-OST0000 complete [ 3557.893750] Lustre: server umount lustre-OST0001 complete [ 3568.967930] Lustre: DEBUG MARKER: == conf-sanity test 88a: tunefs.lustre persists a --mountfsoptions-only change ========================================================== 10:02:02 (1788444122) [ 3592.382169] Lustre: DEBUG MARKER: == conf-sanity test 88b: check the default mount options can be overridden ========================================================== 10:02:26 (1788444146) [ 3602.224316] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing load_modules_local [ 3622.152437] Lustre: DEBUG MARKER: == conf-sanity test 89: check tunefs --param and --erase-param{s} options ========================================================== 10:02:56 (1788444176) [ 3663.377765] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_hostid [ 3673.521831] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing load_modules_local [ 3711.359467] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3711.607716] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3711.638146] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3711.757769] Lustre: lustre-MDT0000: new disk, initializing [ 3711.856106] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3711.874600] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3716.574789] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 3727.447124] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3727.571361] Lustre: 126550:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 3727.649784] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 3727.654680] Lustre: Skipped 1 previous similar message [ 3727.751752] Lustre: lustre-MDT0001: new disk, initializing [ 3727.825664] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3727.878848] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 3727.889692] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3733.205233] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 3741.559689] Lustre: DEBUG MARKER: oleg248-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3746.750393] Lustre: DEBUG MARKER: oleg248-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3751.650199] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3751.925486] Lustre: lustre-OST0000: new disk, initializing [ 3751.934909] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3751.945318] Lustre: 128518:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3752.057367] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3757.437966] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 3760.712763] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 3760.738824] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 3760.869603] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3766.072810] Lustre: DEBUG MARKER: oleg248-client.virtnet: executing wait_import_state_mount (FULL|IDLE) failover.node=192.0.2.254@tcp0,osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3770.705932] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 3770.938209] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 3775.766297] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 3776.038825] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 3781.603300] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3781.607151] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3781.614393] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3781.622803] Lustre: Skipped 1 previous similar message [ 3781.641756] Lustre: Skipped 1 previous similar message [ 3786.722379] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3786.725127] Lustre: Skipped 1 previous similar message [ 3791.840702] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3791.865438] Lustre: Skipped 1 previous similar message [ 3793.376849] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 3793.617722] Lustre: server umount lustre-OST0000 complete [ 3799.524759] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3799.530950] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3802.080260] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3802.081552] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3802.116670] Lustre: Skipped 2 previous similar messages [ 3804.571111] LustreError: 130061:0:(ldlm_resource.c:1207:ldlm_resource_complain()) MGS: namespace resource [0x736d61726170:0x3:0x0].0x0 (00000000cfe8bdae) refcount nonzero (1) after lock cleanup; forcing cleanup. [ 3806.663578] Lustre: server umount lustre-MDT0000 complete [ 3810.951584] LustreError: 126542:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788444366 with bad export cookie 12918797705063487198 [ 3810.960792] LustreError: MGC192.168.202.148@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3810.970122] LustreError: 126542:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 3811.486535] Lustre: server umount lustre-MDT0001 complete [ 3823.413609] Lustre: DEBUG MARKER: == conf-sanity test 90a: check max_mod_rpcs_in_flight is enforced ========================================================== 10:06:17 (1788444377) [ 3831.099237] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3831.642267] LustreError: 130935:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3831.659894] LustreError: 130935:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [ 3831.749576] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3836.899629] LustreError: 130936:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3837.704708] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 3841.003406] LustreError: 130935:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3846.133476] LustreError: 130936:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3847.285054] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3852.853822] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 3860.821591] Lustre: DEBUG MARKER: oleg248-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3866.984848] Lustre: DEBUG MARKER: oleg248-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3872.966581] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3873.315543] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3873.327597] Lustre: Skipped 1 previous similar message [ 3879.897812] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 3887.342157] Lustre: DEBUG MARKER: oleg248-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3891.435829] Lustre: *** cfs_fail_loc=159, val=0*** [ 3907.666328] Lustre: lustre-MDT0000: Client f10999ef-f50d-47c2-b127-49f2e6a6c166 (at 192.168.202.48@tcp) reconnecting [ 3909.893559] Lustre: *** cfs_fail_loc=159, val=0*** [ 3909.910331] Lustre: Skipped 8 previous similar messages [ 3925.571182] Lustre: lustre-MDT0000: Client f10999ef-f50d-47c2-b127-49f2e6a6c166 (at 192.168.202.48@tcp) reconnecting [ 3932.644792] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3932.655227] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3932.673488] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3932.682395] Lustre: Skipped 1 previous similar message [ 3932.696757] Lustre: Skipped 1 previous similar message [ 3933.208931] Lustre: server umount lustre-OST0000 complete [ 3943.455755] LustreError: 133957:0:(ldlm_resource.c:1207:ldlm_resource_complain()) MGS: namespace resource [0x736d61726170:0x3:0x0].0x0 (00000000022395d2) refcount nonzero (1) after lock cleanup; forcing cleanup. [ 3943.620136] Lustre: server umount lustre-MDT0000 complete [ 3947.127650] LustreError: 133656:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788444502 with bad export cookie 12918797705063488080 [ 3947.138100] LustreError: MGC192.168.202.148@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3947.412683] Lustre: server umount lustre-MDT0001 complete [ 3958.326839] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing unload_modules_local [ 3961.760948] Key type lgssc unregistered [ 3962.121388] LNet: 134712:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3962.130242] LNetError: 134712:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3962.150718] LNet: Removed LNI 192.168.202.148@tcp [ 3963.021106] Key type .llcrypt unregistered [ 3963.023074] Key type ._llcrypt unregistered [ 3987.591330] Key type ._llcrypt registered [ 3987.593028] Key type .llcrypt registered [ 3987.751779] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing load_modules_local [ 3988.934648] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3989.220484] alg: No test for adler32 (adler32-zlib) [ 3990.388497] Lustre: Lustre: Build Version: 2.17.56_51_g2aa7cea [ 3990.750908] LNet: Added LNI 192.168.202.148@tcp [8/256/0/180] [ 3992.439397] Key type lgssc registered [ 3993.757700] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4002.381478] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4002.397799] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4003.804255] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4007.634075] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 4013.026283] LustreError: 136126:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4013.046716] LustreError: 136126:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 4017.954898] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4018.145745] LustreError: 136127:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4018.614093] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4024.519130] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 4033.096581] Lustre: DEBUG MARKER: oleg248-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4039.280499] Lustre: DEBUG MARKER: oleg248-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4046.527431] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4046.780302] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4048.821217] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:42 to 0x280000401:65) [ 4052.088993] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 4058.819567] Lustre: DEBUG MARKER: oleg248-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4072.178924] Lustre: *** cfs_fail_loc=159, val=0*** [ 4088.404291] Lustre: lustre-MDT0000: Client f6bf408c-68ea-4d8a-aa14-c5ec24fb3370 (at 192.168.202.48@tcp) reconnecting [ 4091.366511] Lustre: *** cfs_fail_loc=159, val=0*** [ 4107.841108] Lustre: lustre-MDT0001: Client f6bf408c-68ea-4d8a-aa14-c5ec24fb3370 (at 192.168.202.48@tcp) reconnecting [ 4109.393839] Lustre: *** cfs_fail_loc=159, val=0*** [ 4109.400488] Lustre: Skipped 3 previous similar messages [ 4125.750948] Lustre: lustre-MDT0001: Client f6bf408c-68ea-4d8a-aa14-c5ec24fb3370 (at 192.168.202.48@tcp) reconnecting [ 4130.724265] Lustre: 139162:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4134.209453] Lustre: *** cfs_fail_loc=159, val=0*** [ 4134.211305] Lustre: Skipped 4 previous similar messages [ 4150.877923] Lustre: lustre-MDT0000: Client 5e7b1ca0-d8a9-44e3-b345-7ed264aea446 (at 192.168.202.48@tcp) reconnecting [ 4153.083923] Lustre: *** cfs_fail_loc=159, val=0*** [ 4153.089546] Lustre: Skipped 14 previous similar messages [ 4168.262273] Lustre: lustre-MDT0000: Client 5e7b1ca0-d8a9-44e3-b345-7ed264aea446 (at 192.168.202.48@tcp) reconnecting [ 4169.998229] Lustre: 139457:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4175.848574] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4175.850035] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4175.859240] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4175.878647] Lustre: Skipped 1 previous similar message [ 4180.960791] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4180.985186] Lustre: Skipped 2 previous similar messages [ 4181.400198] Lustre: server umount lustre-OST0000 complete [ 4186.087663] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4186.088920] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4186.094854] Lustre: Skipped 1 previous similar message [ 4186.110489] Lustre: Skipped 1 previous similar message [ 4190.786253] Lustre: server umount lustre-MDT0000 complete [ 4191.202281] LustreError: 139224:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4191.222261] LustreError: 139224:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 4194.606849] LustreError: 136107:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788444749 with bad export cookie 18343444624576196873 [ 4194.622134] LustreError: MGC192.168.202.148@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4194.636810] LustreError: 136107:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4195.446994] Lustre: server umount lustre-MDT0001 complete [ 4207.046636] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing unload_modules_local [ 4209.528790] Key type lgssc unregistered [ 4209.774448] LNet: 140514:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4209.782439] LNetError: 140514:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4209.823339] LNet: Removed LNI 192.168.202.148@tcp [ 4210.956122] Key type .llcrypt unregistered [ 4210.964393] Key type ._llcrypt unregistered [ 4240.592699] Key type ._llcrypt registered [ 4240.594446] Key type .llcrypt registered [ 4240.734709] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing load_modules_local [ 4242.038452] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4242.058302] alg: No test for adler32 (adler32-zlib) [ 4243.340365] Lustre: Lustre: Build Version: 2.17.56_51_g2aa7cea [ 4243.647497] LNet: Added LNI 192.168.202.148@tcp [8/256/0/180] [ 4245.415145] Key type lgssc registered [ 4246.444674] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4254.102470] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4254.132404] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4255.664461] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4260.993933] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 4264.937267] LustreError: 141928:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4264.958113] LustreError: 141928:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 4269.942319] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4270.056741] LustreError: 141929:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4270.274720] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4274.883917] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 4282.320800] Lustre: DEBUG MARKER: oleg248-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4288.319648] Lustre: DEBUG MARKER: oleg248-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4295.950263] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4296.314794] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4300.409741] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:129) [ 4303.106678] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 4310.991641] Lustre: DEBUG MARKER: oleg248-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4315.773847] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:97) [ 4319.736580] Lustre: 144512:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4322.896219] Lustre: 144561:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4324.157564] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 in log params [ 4325.109533] Lustre: Setting parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=8 in log params [ 4329.128021] Lustre: 144706:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre/mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 (mode = 0) failed: rc = -17 [ 4330.338169] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=7 in log params [ 4331.283307] Lustre: 144805:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4336.614248] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4336.618226] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4336.627336] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4336.645710] Lustre: Skipped 1 previous similar message [ 4341.732581] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4341.742021] Lustre: Skipped 2 previous similar messages [ 4342.760052] Lustre: server umount lustre-OST0000 complete [ 4346.857603] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4346.860987] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4346.881081] Lustre: Skipped 1 previous similar message [ 4346.889766] Lustre: Skipped 1 previous similar message [ 4351.974141] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4351.994802] Lustre: Skipped 1 previous similar message [ 4353.204687] Lustre: server umount lustre-MDT0000 complete [ 4357.034176] LustreError: 141909:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788444912 with bad export cookie 2515473117456456563 [ 4357.042570] LustreError: MGC192.168.202.148@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4357.046720] LustreError: 141909:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 4358.275174] Lustre: server umount lustre-MDT0001 complete [ 4371.422563] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing unload_modules_local [ 4375.110459] Key type lgssc unregistered [ 4375.507882] LNet: 145868:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4375.525320] LNetError: 145868:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4375.588070] LNet: Removed LNI 192.168.202.148@tcp [ 4376.866905] Key type .llcrypt unregistered [ 4376.869190] Key type ._llcrypt unregistered [ 4412.339271] Key type ._llcrypt registered [ 4412.341259] Key type .llcrypt registered [ 4412.511635] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing load_modules_local [ 4413.936428] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4414.101543] alg: No test for adler32 (adler32-zlib) [ 4415.371863] Lustre: Lustre: Build Version: 2.17.56_51_g2aa7cea [ 4415.607419] LNet: Added LNI 192.168.202.148@tcp [8/256/0/180] [ 4417.247211] Key type lgssc registered [ 4418.040446] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4425.491326] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4425.506609] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4427.017440] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4431.485362] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 4437.486544] LustreError: 147281:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4437.518970] LustreError: 147281:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 4442.597271] LustreError: 147282:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4442.623794] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4443.063674] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4449.012393] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 4457.516370] Lustre: DEBUG MARKER: oleg248-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4464.816879] Lustre: DEBUG MARKER: oleg248-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4472.485528] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4472.749314] Lustre: lustre-OST0000: Not available for connect from 0@lo (not set up) [ 4472.857707] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4477.948840] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:161) [ 4480.205354] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 4487.738670] Lustre: DEBUG MARKER: oleg248-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4488.187357] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:129) [ 4494.549432] Lustre: *** cfs_fail_loc=159, val=0*** [ 4510.339961] Lustre: lustre-MDT0000: Client 86c3aec2-f137-4b67-8a38-9e45aafc847c (at 192.168.202.48@tcp) reconnecting [ 4518.888365] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4518.888821] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4518.904083] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4518.923947] LustreError: Skipped 1 previous similar message [ 4522.620642] Lustre: server umount lustre-OST0000 complete [ 4529.123097] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4529.123761] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4529.141622] Lustre: Skipped 2 previous similar messages [ 4529.165875] Lustre: Skipped 2 previous similar messages [ 4532.936968] Lustre: server umount lustre-MDT0000 complete [ 4534.241380] LustreError: 147281:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4534.269120] LustreError: 147281:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 4536.036573] LustreError: 147262:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788445091 with bad export cookie 9729815757878110189 [ 4536.041675] LustreError: MGC192.168.202.148@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4536.042305] LustreError: 147262:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4536.320662] Lustre: server umount lustre-MDT0001 complete [ 4546.516139] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing unload_modules_local [ 4549.585155] Key type lgssc unregistered [ 4549.855850] LNet: 150950:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4549.870399] LNetError: 150950:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4549.897194] LNet: Removed LNI 192.168.202.148@tcp [ 4550.892218] Key type .llcrypt unregistered [ 4550.894653] Key type ._llcrypt unregistered [ 4575.953765] Key type ._llcrypt registered [ 4575.957516] Key type .llcrypt registered [ 4576.086411] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing load_modules_local [ 4577.556821] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4577.868457] alg: No test for adler32 (adler32-zlib) [ 4578.971866] Lustre: Lustre: Build Version: 2.17.56_51_g2aa7cea [ 4579.419371] LNet: Added LNI 192.168.202.148@tcp [8/256/0/180] [ 4581.136097] Key type lgssc registered [ 4582.757349] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4593.512496] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4593.554650] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4594.924858] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4600.331279] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 4605.412204] LustreError: 152365:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4605.435883] LustreError: 152365:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 4608.853196] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4609.288717] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4614.610661] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 4622.860683] Lustre: DEBUG MARKER: oleg248-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4629.355624] Lustre: DEBUG MARKER: oleg248-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4636.735489] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4637.350204] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4640.472815] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:170 to 0x280000401:193) [ 4644.601789] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 4652.053561] Lustre: DEBUG MARKER: oleg248-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4654.776532] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:161) [ 4676.990050] Lustre: lustre-OST0000: evicting a2a3e1b6-f95c-4be5-bd30-a7b1f05432c0 (at 192.168.202.48@tcp) by administrative request [ 4677.006561] Lustre: Skipped 1 previous similar message [ 4707.298988] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4707.310910] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4707.347737] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4711.399503] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4711.402156] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4711.416696] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4716.515141] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4716.520761] Lustre: Skipped 1 previous similar message [ 4721.119399] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 4721.358052] Lustre: server umount lustre-OST0000 complete [ 4726.755398] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4726.777542] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4726.784698] Lustre: Skipped 1 previous similar message [ 4731.877689] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4731.893124] Lustre: Skipped 1 previous similar message [ 4732.852665] Lustre: server umount lustre-MDT0000 complete [ 4736.994239] LustreError: 152366:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4737.004656] LustreError: 152366:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 4737.292173] LustreError: 155684:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788445292 with bad export cookie 8673600510645454508 [ 4737.299898] LustreError: MGC192.168.202.148@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4737.309889] LustreError: 155684:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 4737.634795] Lustre: server umount lustre-MDT0001 complete [ 4750.766625] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing unload_modules_local [ 4754.125470] Key type lgssc unregistered [ 4754.387636] LNet: 156439:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4754.392377] LNetError: 156439:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4755.430236] LNet: Removed LNI 192.168.202.148@tcp [ 4756.662244] Key type .llcrypt unregistered [ 4756.664983] Key type ._llcrypt unregistered [ 4834.240235] Key type ._llcrypt registered [ 4834.245156] Key type .llcrypt registered [ 4834.334931] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing load_modules_local [ 4836.351796] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4836.385034] alg: No test for adler32 (adler32-zlib) [ 4837.620371] Lustre: Lustre: Build Version: 2.17.56_51_g2aa7cea [ 4837.908320] LNet: Added LNI 192.168.202.148@tcp [8/256/0/180] [ 4839.623277] Key type lgssc registered [ 4840.708265] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4848.468183] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4848.481929] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4849.903664] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4854.573986] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 4860.385367] LustreError: 158932:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4860.403462] LustreError: 158932:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 4861.409426] LustreError: 158933:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4863.311123] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4863.609784] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4867.861499] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 4875.767521] Lustre: DEBUG MARKER: oleg248-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4882.338854] Lustre: DEBUG MARKER: oleg248-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4891.560739] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4892.385348] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4896.534345] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:195 to 0x280000401:225) [ 4903.278906] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 4907.861817] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:193) [ 4913.273925] Lustre: DEBUG MARKER: oleg248-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4927.460678] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4927.478540] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4927.516192] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4928.488893] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4928.506817] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4928.507844] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4930.681300] Lustre: server umount lustre-OST0000 complete [ 4938.723278] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4938.732623] Lustre: Skipped 1 previous similar message [ 4938.734028] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4938.740738] Lustre: Skipped 1 previous similar message [ 4941.709284] Lustre: server umount lustre-MDT0000 complete [ 4943.850707] LustreError: 158933:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4943.862097] LustreError: 158933:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 4946.381857] LustreError: 158914:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788445501 with bad export cookie 15685231980889008521 [ 4946.390287] LustreError: MGC192.168.202.148@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4946.399354] LustreError: 158914:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4946.911512] Lustre: server umount lustre-MDT0001 complete [ 4959.435567] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing unload_modules_local [ 4964.036339] Key type lgssc unregistered [ 4964.601816] LNet: 162478:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4964.611044] LNetError: 162478:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4964.632277] LNet: Removed LNI 192.168.202.148@tcp [ 4966.013579] Key type .llcrypt unregistered [ 4966.015819] Key type ._llcrypt unregistered [ 4994.867863] Key type ._llcrypt registered [ 4994.873341] Key type .llcrypt registered [ 4994.986565] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing load_modules_local [ 4996.588504] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4996.626762] alg: No test for adler32 (adler32-zlib) [ 4997.724459] Lustre: Lustre: Build Version: 2.17.56_51_g2aa7cea [ 4998.036613] LNet: Added LNI 192.168.202.148@tcp [8/256/0/180] [ 4999.775373] Key type lgssc registered [ 5001.280658] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5018.774882] Lustre: DEBUG MARKER: == conf-sanity test 100: check lshowmount lists MGS, MDT, OST and 0@lo ========================================================== 10:26:12 (1788445572) [ 5035.390140] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_hostid [ 5049.377282] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing load_modules_local [ 5091.261950] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5091.309782] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5092.771647] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 5092.798242] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 5092.935477] Lustre: lustre-MDT0000: new disk, initializing [ 5093.081726] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5093.125262] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 5098.511807] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 5111.181971] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5111.375446] Lustre: 166742:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 5111.433085] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 5111.440314] Lustre: Skipped 1 previous similar message [ 5111.548495] Lustre: lustre-MDT0001: new disk, initializing [ 5111.683656] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5111.752085] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 5111.772895] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 5117.244296] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 5127.602974] Lustre: DEBUG MARKER: oleg248-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 5134.838429] Lustre: DEBUG MARKER: oleg248-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 5142.875406] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5143.123436] Lustre: lustre-OST0000: new disk, initializing [ 5143.136742] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 5143.145843] Lustre: 168713:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 5143.241139] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5150.818875] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 5150.909036] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 5150.944176] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 5151.064169] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 5161.483437] Lustre: DEBUG MARKER: oleg248-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 5176.810042] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5176.817124] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 5176.826280] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5176.828497] Lustre: Skipped 1 previous similar message [ 5176.845781] LustreError: Skipped 1 previous similar message [ 5178.649387] Lustre: server umount lustre-OST0000 complete [ 5183.460880] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 5183.485715] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5183.518564] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5183.526608] Lustre: Skipped 1 previous similar message [ 5187.040618] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5187.043505] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5192.161369] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5192.166206] Lustre: Skipped 2 previous similar messages [ 5197.279157] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 5197.282093] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5197.298658] Lustre: Skipped 1 previous similar message [ 5197.500556] Lustre: server umount lustre-MDT0000 complete [ 5201.275065] LustreError: 166734:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788445756 with bad export cookie 7843534151271395390 [ 5201.281513] LustreError: MGC192.168.202.148@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5201.285082] LustreError: 166734:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 5201.643545] Lustre: server umount lustre-MDT0001 complete [ 5212.705976] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing unload_modules_local [ 5214.810530] Key type lgssc unregistered [ 5215.130797] LNet: 170624:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5215.140182] LNetError: 170624:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5215.165161] LNet: Removed LNI 192.168.202.148@tcp [ 5215.889569] Key type .llcrypt unregistered [ 5215.891792] Key type ._llcrypt unregistered [ 5241.483888] Key type ._llcrypt registered [ 5241.486525] Key type .llcrypt registered [ 5241.615526] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing load_modules_local [ 5242.655712] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5242.699534] alg: No test for adler32 (adler32-zlib) [ 5244.075654] Lustre: Lustre: Build Version: 2.17.56_51_g2aa7cea [ 5244.483045] LNet: Added LNI 192.168.202.148@tcp [8/256/0/180] [ 5246.287710] Key type lgssc registered [ 5247.694743] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5261.143949] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5261.179448] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5262.811040] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5268.179965] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 5272.035761] LustreError: 172040:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5272.063399] LustreError: 172040:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 5276.674601] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5276.961674] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5281.690415] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 5289.333556] Lustre: DEBUG MARKER: oleg248-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 5295.167198] Lustre: DEBUG MARKER: oleg248-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 5302.267362] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5302.735713] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5307.830911] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:3 to 0x280000401:33) [ 5309.490142] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 5317.985924] Lustre: DEBUG MARKER: oleg248-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 5325.375518] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5325.391537] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5325.403514] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5325.415836] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5325.441268] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5326.495437] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5326.512613] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5326.531049] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5326.545363] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5326.554133] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5327.527697] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5327.539322] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5327.549589] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5327.557372] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5327.571658] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5328.451389] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5329.425814] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5330.291354] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5330.295560] Lustre: Skipped 2 previous similar messages [ 5330.309469] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5330.316458] Lustre: Skipped 2 previous similar messages [ 5330.319962] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5330.322340] Lustre: Skipped 2 previous similar messages [ 5330.325423] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5330.334683] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5330.340964] Lustre: Skipped 2 previous similar messages [ 5331.127789] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5332.181620] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5333.289896] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5334.303207] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5334.319702] Lustre: Skipped 3 previous similar messages [ 5334.336430] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5334.343741] Lustre: Skipped 3 previous similar messages [ 5334.353089] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5334.364438] Lustre: Skipped 3 previous similar messages [ 5334.375840] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5334.387030] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5334.398928] Lustre: Skipped 3 previous similar messages [ 5335.659837] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5336.648587] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5337.573941] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5338.410151] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5339.333703] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5340.222269] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5341.086580] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5341.852435] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5342.799134] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5342.804805] Lustre: Skipped 8 previous similar messages [ 5342.815902] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5342.823598] Lustre: Skipped 8 previous similar messages [ 5342.830046] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5342.832975] Lustre: Skipped 8 previous similar messages [ 5342.836724] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5342.846713] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5342.854910] Lustre: Skipped 8 previous similar messages [ 5343.773358] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5344.708392] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5345.430714] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5346.328049] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5347.277584] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5348.221455] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5349.243327] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5350.103750] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5350.918714] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5351.764801] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5352.762794] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5353.649187] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5354.610348] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5355.629108] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5356.640924] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5357.621197] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5358.300508] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5359.022631] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5359.028606] Lustre: Skipped 17 previous similar messages [ 5359.045103] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5359.052574] Lustre: Skipped 17 previous similar messages [ 5359.055912] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5359.060184] Lustre: Skipped 17 previous similar messages [ 5359.063170] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5359.069989] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5359.081174] Lustre: Skipped 17 previous similar messages [ 5359.997402] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5361.074814] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5362.206473] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5363.370518] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5364.113628] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5364.909169] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5365.796741] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5366.711180] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5367.658272] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5368.780257] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5369.870874] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5370.852233] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5372.087311] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5372.842191] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5373.836921] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5374.835095] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5375.949986] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5377.243077] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5378.226342] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5379.580711] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5380.817577] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5381.743029] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5383.042912] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5383.975467] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5384.820533] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5385.636538] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5386.528531] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5387.620547] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5388.527400] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5389.442619] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5390.261701] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5390.975944] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5391.834970] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5391.837901] Lustre: Skipped 32 previous similar messages [ 5391.857839] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5391.866156] Lustre: Skipped 32 previous similar messages [ 5391.869573] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5391.874971] Lustre: Skipped 32 previous similar messages [ 5391.884892] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5391.895595] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5391.900811] Lustre: Skipped 32 previous similar messages [ 5392.929280] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5393.888542] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5394.736843] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5395.644326] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5396.520782] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5397.533052] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5398.489141] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5399.387689] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5400.168102] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5401.156622] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5401.997530] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5402.992720] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5403.902653] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5404.838266] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5405.678358] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5406.804223] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5407.831854] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5408.862231] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5409.724795] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5410.681518] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5411.418135] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5412.547470] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5413.394097] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5414.288537] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5415.321034] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5416.056624] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5417.003803] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5418.046864] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5418.904982] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5420.014594] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5420.859331] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5421.830274] LustreError: 172060:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5421.856290] LustreError: 172060:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5421.873396] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5422.688220] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5423.608625] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5424.431686] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5425.479588] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5426.491385] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5427.590912] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5428.613432] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5429.668252] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5431.285734] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5432.278300] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5433.370127] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5434.406685] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5435.190954] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5436.383629] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5437.214823] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5438.535539] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5439.634252] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5440.602735] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5441.656432] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5442.718706] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5443.409756] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5444.093567] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5444.841634] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5446.279928] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5447.074383] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5447.754862] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5448.486319] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5449.117204] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5450.049423] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5450.935076] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5452.019231] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5452.944593] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5453.864473] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5454.751907] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5455.902091] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5455.908856] Lustre: Skipped 66 previous similar messages [ 5455.929868] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5455.938339] Lustre: Skipped 66 previous similar messages [ 5455.943232] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5455.949185] Lustre: Skipped 66 previous similar messages [ 5455.953733] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5455.965193] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5455.974822] Lustre: Skipped 66 previous similar messages [ 5456.832688] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5457.850437] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5458.579215] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5459.523193] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5460.373417] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5461.112284] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5461.999796] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5462.843090] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5463.735966] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5464.531884] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5465.491267] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5466.532959] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5467.548574] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5468.655655] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5469.622384] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5470.616123] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5471.515251] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5472.522143] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5473.325761] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5474.307047] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5475.283749] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5476.239528] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5476.980054] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5477.856582] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5478.637114] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5479.523333] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5480.327574] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5481.138473] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5481.889389] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5482.767045] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5483.644647] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5484.542745] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5485.546833] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5486.377702] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5487.487037] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5488.488703] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5489.275364] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5490.087952] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5490.917784] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5491.895494] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5492.969219] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5494.066809] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5494.979395] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5496.054599] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5497.154095] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5497.980790] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5499.034910] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5499.951587] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5500.980122] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5501.876662] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5502.809917] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5503.772899] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5504.799799] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5505.840772] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5507.055920] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5507.902910] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5508.672275] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5509.550729] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5510.597446] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5511.543819] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5512.625534] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5513.491449] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5514.448384] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5515.524987] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5516.426325] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5517.363825] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5518.567343] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5519.524315] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5520.785885] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5521.895937] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5522.938683] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5523.855706] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5524.702934] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5525.704842] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5526.597717] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5527.820579] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5528.824688] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5529.758312] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5530.570220] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5531.513158] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5532.419199] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5533.668798] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5534.936993] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5536.287883] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5537.090803] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5538.002234] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5538.986109] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5540.357946] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5541.234424] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5542.522649] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5543.421315] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5544.304713] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5545.476344] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5546.531634] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5547.865870] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5548.792130] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5549.779884] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5550.645883] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5551.954388] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5553.003603] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5554.312232] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5555.072626] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5556.404970] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5557.873716] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5558.484518] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5559.446555] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5560.463523] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5561.800556] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5562.952832] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5564.118392] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5565.451983] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5566.483687] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5567.392460] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5568.281582] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5569.251986] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5570.198578] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5571.282398] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5572.651473] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5573.637823] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5574.744882] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5575.632217] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5576.753423] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5577.668840] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5578.677525] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5579.616535] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5580.544945] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5581.484385] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5582.492667] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5583.606476] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5584.668572] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5584.675431] Lustre: Skipped 129 previous similar messages [ 5584.701758] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5584.716260] Lustre: Skipped 129 previous similar messages [ 5584.720552] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5584.726273] Lustre: Skipped 129 previous similar messages [ 5584.731061] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5584.739766] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5584.742634] Lustre: Skipped 129 previous similar messages [ 5585.598435] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5586.626135] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5587.557336] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5588.519685] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5589.681478] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5590.705696] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5591.828023] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5592.841575] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5593.769898] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5594.878499] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5595.922448] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5596.843940] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5597.688350] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5598.569397] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5599.551910] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5600.518079] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5601.570759] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5602.678823] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5603.828255] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5604.854843] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5605.960516] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5606.957275] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5607.870367] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5608.910959] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5609.866663] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5611.074463] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5611.994274] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5612.928297] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5613.662572] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5614.720893] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5615.703427] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5616.629072] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5617.645758] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5618.685707] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5619.494283] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5620.499807] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5621.335849] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5622.262448] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5623.236691] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5624.456095] LustreError: 172060:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5624.471199] LustreError: 172060:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5624.478256] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5625.306069] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5626.287415] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5627.140773] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5628.223387] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5629.194833] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5630.090148] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5630.809434] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5631.694641] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5632.750180] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5633.541969] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5634.314052] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5635.222992] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5636.030411] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5637.059281] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5638.025967] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5639.040971] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5640.031419] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5641.097168] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5642.240634] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5643.075777] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5643.980817] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5644.684466] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5645.484962] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5646.288213] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5647.038698] LustreError: 172060:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5647.045712] LustreError: 172060:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5647.049116] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5647.767245] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5648.785221] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5649.866301] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5650.985860] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5651.844925] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5652.761912] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5653.542667] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5654.345624] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5655.204842] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5656.249453] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5657.100555] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5658.094282] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5658.884146] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5659.758153] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5660.572272] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5661.482260] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5662.284988] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5663.263526] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5664.146589] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5665.174535] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5666.260887] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5667.494512] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5668.462765] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5669.416092] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5670.368520] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5671.539645] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5672.557377] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5673.469045] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5674.456680] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5675.511448] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5676.556435] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5677.342635] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5678.290858] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5679.281304] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5680.188494] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5681.053667] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5681.802882] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5682.648544] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5683.509725] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5684.444833] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5685.391750] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5686.274116] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5687.263282] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5688.161328] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5689.201942] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5690.154411] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5691.143494] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5691.961286] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5693.037546] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5693.959858] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5694.919921] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5695.802436] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5696.855778] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5697.666882] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5698.787638] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5699.681952] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5700.614565] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5701.489400] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5702.507580] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5703.437536] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5704.374413] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5705.288083] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5706.230231] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5707.133731] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5708.092965] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5708.973725] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5709.891322] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5710.809689] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5711.867799] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5712.797484] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5713.675205] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5714.560607] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5715.532031] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5716.381542] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5717.297762] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5718.033682] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5718.892850] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5719.868496] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5720.777551] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5721.544965] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5722.982519] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5724.467583] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5725.569709] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5726.773605] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5727.679571] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5728.758988] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5729.655763] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5730.586447] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5731.687782] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5732.647426] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5733.664870] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5734.754356] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5735.720269] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5736.632580] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5737.577078] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5738.615075] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5739.528901] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5740.425604] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5741.400058] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5742.334643] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5743.259757] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5744.316267] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5745.249725] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5746.213146] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5747.091892] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5747.990548] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5748.963247] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5749.914393] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5751.021534] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5751.983587] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5752.973813] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5753.900927] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5754.992550] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5756.022846] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5757.060577] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5757.904230] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5759.057036] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5759.920622] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5760.849590] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5761.680610] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5762.609581] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5763.479156] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5764.439767] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5765.294671] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5766.372349] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5767.281472] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5768.335212] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5769.266197] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5770.339920] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5771.133976] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5772.027944] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5772.840720] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5773.692600] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5774.291924] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5775.045746] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5775.825747] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5776.850746] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5777.931208] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5778.779585] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5779.897768] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5780.802121] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5781.961472] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5782.887775] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5784.095076] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5784.974801] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5786.137894] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5786.849688] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5787.613769] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5788.770757] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5789.747421] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5790.619127] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5791.883266] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5793.098830] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5794.298195] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5795.271322] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5796.555880] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5797.792814] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5799.035849] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5800.012813] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5801.418227] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5802.681811] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5803.793274] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5804.991800] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5806.152849] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5807.077589] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5808.103950] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5808.993159] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5809.755306] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5810.343179] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5811.190162] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5812.249166] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5813.277973] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5814.248787] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5815.168867] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5816.398517] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5817.504333] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5818.609816] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5819.591936] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5820.708944] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5821.992550] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5823.067824] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5824.144178] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5825.071486] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5826.000449] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5826.948564] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5828.234387] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5829.411186] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5830.631932] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5831.634562] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5832.855499] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5834.106599] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5835.082847] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5835.934717] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5836.854576] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5837.872432] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5838.871200] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5839.745726] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5840.726655] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5840.735434] Lustre: Skipped 262 previous similar messages [ 5840.755648] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5840.773574] Lustre: Skipped 262 previous similar messages [ 5840.778279] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5840.781157] Lustre: Skipped 262 previous similar messages [ 5840.789483] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5840.799299] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5840.811584] Lustre: Skipped 262 previous similar messages [ 5841.743795] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5842.636121] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5843.740420] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5844.717553] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5845.715791] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5846.761154] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5847.717717] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5848.758875] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5849.527574] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5850.693134] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5851.670118] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5852.562827] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5853.544465] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5854.746599] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5855.768281] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5856.782414] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5857.730414] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5858.858268] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5859.653654] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5860.653334] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5861.481364] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5862.251726] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5863.155322] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5864.223864] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5865.015082] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5866.155375] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5867.145956] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5867.936915] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5868.789583] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5869.883774] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5871.000842] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5871.944249] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5872.981323] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5873.797891] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5874.935527] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5875.776990] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5876.771713] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5877.752978] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5878.669382] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5879.560337] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5880.757592] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5881.724132] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5882.887512] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5883.735356] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5884.698443] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5885.495167] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5886.492847] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5887.596107] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5888.473374] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5889.647590] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5890.519551] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5891.429556] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5892.362267] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5893.194774] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5894.509728] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5895.432726] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5896.486809] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5897.703973] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5898.881782] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5899.628744] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5900.669234] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5901.379775] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5902.207205] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5903.092692] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5904.066792] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5904.859924] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5905.956286] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5907.075927] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5908.196868] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5909.440974] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5910.622412] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5911.607542] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5912.911080] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5913.677487] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5914.772497] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5915.563417] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5916.657917] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5917.568252] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5919.050761] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5920.083862] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5921.198395] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5922.162528] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5923.234979] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5924.119573] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5924.931315] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5925.834069] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5926.866219] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5927.707765] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5928.597530] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5929.556302] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5930.302601] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5931.052441] LustreError: 172060:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5931.065600] LustreError: 172060:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5931.081236] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5931.865158] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5932.518979] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5933.435271] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5934.508615] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5935.186252] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5936.058126] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5936.853137] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5937.870420] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5938.813987] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5939.737210] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5940.568598] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5941.659583] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5942.606736] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5943.773148] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5944.599934] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5945.799171] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5946.859734] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5947.786689] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5948.869756] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5949.611503] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5950.497746] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5951.287469] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5952.332522] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5953.201807] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5954.203729] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5955.013582] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5956.327383] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5957.684427] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5958.733280] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5959.658872] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5960.503575] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5961.481318] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5962.465385] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5963.626804] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5964.412913] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5965.402685] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5966.365973] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5967.294983] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5968.523174] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5969.330832] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5970.405949] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5971.210540] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5972.028312] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5972.883432] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5973.739840] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5974.666842] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5975.712895] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5976.743784] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5977.656720] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5978.775565] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5979.540030] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5980.616293] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5981.528800] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5982.498312] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5983.394340] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5984.511566] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5985.691267] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5986.606378] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5987.702081] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5988.529148] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5989.439178] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5990.407585] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5991.504504] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5992.346257] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5993.408081] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5994.454386] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5995.643476] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5996.647266] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5997.742576] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5998.782597] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5999.692089] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6000.655215] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6001.724169] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6002.647956] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6003.738806] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6004.784187] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6005.994373] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6007.092931] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6008.274419] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6009.549142] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6010.622692] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6011.696664] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6012.663869] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6013.707716] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6014.594694] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6015.569647] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6016.419899] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6017.273260] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6018.153191] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6018.993238] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6019.919607] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6020.935737] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6021.896115] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6022.798579] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6023.909530] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6024.899100] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6026.079806] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6027.143634] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6028.224632] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6029.375971] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6030.471626] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6031.733797] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6032.729334] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6033.605677] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6034.495972] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6035.461168] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6036.973540] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6037.838170] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6038.978813] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6040.269234] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6041.288531] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6042.653739] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6043.891844] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6044.859586] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6046.066387] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6047.316387] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6048.403656] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6049.086605] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6049.970096] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6051.013804] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6051.913072] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6052.782572] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6053.896799] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6055.052533] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6055.848070] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6056.587210] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6057.518967] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6058.458513] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6059.313066] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6060.282874] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6061.215922] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6062.514375] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6063.276556] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6064.234984] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6065.191657] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6066.194156] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6067.143265] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6068.316530] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6069.260990] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6070.222550] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6071.129754] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6072.173718] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6073.065303] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6074.013863] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6074.951500] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6075.757143] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6076.741508] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6077.741736] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6078.933826] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6079.807235] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6080.788465] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6081.585172] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6082.397336] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6083.054453] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6083.968596] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6084.692972] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6085.692454] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6086.450573] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6087.339356] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6088.125659] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6088.915978] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6089.745443] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6090.590917] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6091.532903] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6092.505942] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6093.503604] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6094.493691] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6095.465330] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6096.494112] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6097.351547] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6098.235683] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6099.120278] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6100.088763] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6101.151769] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6102.212714] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6103.051734] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6103.998473] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6105.138814] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6106.145395] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6107.331147] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6108.350635] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6109.390438] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6110.284519] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6111.248703] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6112.111584] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6113.327532] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6114.457160] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6115.593754] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6116.466106] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6117.926348] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6119.110607] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6120.211980] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6121.458391] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6122.686396] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6123.922907] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6124.960970] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6125.899963] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6127.064335] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6127.961966] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6129.052556] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6129.923164] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6131.023098] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6132.010132] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6132.984732] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6134.048250] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6134.983793] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6135.862735] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6136.737692] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6137.857485] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6138.856095] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6139.904639] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6140.780402] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6141.678360] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6142.488215] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6143.394324] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6144.262232] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6145.133127] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6146.184090] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6147.183874] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6148.086568] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6149.076804] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6149.971252] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6150.996938] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6151.920593] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6153.034731] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6154.010473] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6154.971067] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6156.059353] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6157.175088] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6158.070117] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6159.261448] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6160.319651] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6161.340488] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6162.527433] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6163.499844] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6164.605795] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6165.457788] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6166.530556] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6167.762845] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6168.766993] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6169.857249] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6170.883828] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6171.788055] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6172.846246] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6173.721681] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6174.738868] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6175.727343] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6176.692571] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6177.848364] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6179.058162] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6180.063807] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6180.864581] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6181.560828] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6182.661827] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6183.618595] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6184.552772] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6185.350277] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6186.427122] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6187.434323] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6188.526461] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6189.423419] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6190.405322] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6191.487302] LustreError: 172060:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 6191.500942] LustreError: 172060:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 6191.509393] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6192.408284] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6193.483793] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6194.689609] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6195.676709] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6196.710264] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6197.554105] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6198.365805] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6199.161483] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6200.039042] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6200.856046] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6201.562750] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6202.314312] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6203.159617] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6204.025128] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6204.915939] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6205.965603] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6206.777603] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6207.661509] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6208.737850] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6209.551934] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6210.412619] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6211.206721] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6212.075704] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6213.098288] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6214.207178] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6215.125224] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6216.327143] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6217.227332] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6218.151218] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6219.290664] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6220.262170] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6221.185501] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6222.137964] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6223.087684] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6223.816422] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6224.710260] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6225.665420] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6226.706335] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6227.639593] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6228.632377] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6229.367112] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6230.166295] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6230.981361] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6232.080199] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6232.947921] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6233.949327] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6234.987914] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6235.805232] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6236.967419] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6237.964698] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6239.061045] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6240.271644] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6241.253837] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6242.333689] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6243.122485] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6244.097277] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6245.143250] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6246.047178] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6246.878866] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6247.800507] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6248.525069] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6249.607606] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6250.397990] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6251.436702] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6252.366658] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6253.469856] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6254.501577] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6255.449621] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6256.390907] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6257.287772] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6258.280189] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6259.300877] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6260.270805] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6261.233495] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6262.177841] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6263.133235] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6264.107295] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6265.505158] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6266.457973] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6267.241926] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6267.950786] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6268.822526] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6269.868334] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6271.039713] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6271.836214] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6272.809738] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6273.822438] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6274.540531] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6275.428077] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6276.857407] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6278.150950] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6279.415070] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6280.473171] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6281.473937] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6282.069889] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6283.281933] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6284.135515] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6284.946081] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6285.703367] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6286.708912] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6288.064117] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6289.384568] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6290.403368] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6291.711493] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6292.837929] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6293.853325] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6294.857349] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6295.712829] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6296.796696] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6297.674717] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6298.475601] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6299.289759] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6300.336330] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6301.190086] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6302.338345] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6628.540393] LustreError: 175684:0:(ofd_dev.c:1780:ofd_create_hdl()) lustre-OST0000: unable to precreate: rc = -28 [ 6628.550330] LustreError: 172060:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -28 [ 7092.720442] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7092.726817] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7092.733538] Lustre: Skipped 471 previous similar messages [ 7092.743917] Lustre: Skipped 1 previous similar message [ 7097.566752] Lustre: server umount lustre-OST0000 complete [ 7102.945510] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7108.067546] LustreError: 172041:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7108.097845] LustreError: 172041:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 7108.163824] Lustre: server umount lustre-MDT0000 complete [ 7112.985498] LustreError: 172775:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788447668 with bad export cookie 2044986629766079662 [ 7112.992537] LustreError: MGC192.168.202.148@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7112.993849] LustreError: 172775:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 7114.123784] Lustre: server umount lustre-MDT0001 complete [ 7131.623176] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing unload_modules_local [ 7135.082434] Key type lgssc unregistered [ 7135.326902] LNet: 227308:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 7135.337044] LNetError: 227308:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 7135.361058] LNet: Removed LNI 192.168.202.148@tcp [ 7136.634703] Key type .llcrypt unregistered [ 7136.637169] Key type ._llcrypt unregistered [ 7164.685430] Key type ._llcrypt registered [ 7164.691621] Key type .llcrypt registered [ 7164.883481] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing load_modules_local [ 7166.072202] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 7166.330099] alg: No test for adler32 (adler32-zlib) [ 7167.472763] Lustre: Lustre: Build Version: 2.17.56_51_g2aa7cea [ 7167.815562] LNet: Added LNI 192.168.202.148@tcp [8/256/0/180] [ 7169.576044] Key type lgssc registered [ 7170.958621] Lustre: Echo OBD driver; http://www.lustre.org/ [ 7180.318972] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 7180.354973] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7182.013817] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7186.941245] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 7192.547740] LustreError: 228726:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7192.575401] LustreError: 228726:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 7195.865448] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7196.385446] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7202.193824] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 7211.034252] Lustre: DEBUG MARKER: oleg248-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 7217.292148] Lustre: DEBUG MARKER: oleg248-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 7225.024216] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7225.467764] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7227.637548] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49665 to 0x280000401:49696) [ 7232.606161] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 7240.349557] Lustre: DEBUG MARKER: oleg248-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 7245.064520] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 7246.758967] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7246.774280] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 7246.781942] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7246.795423] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 7246.800144] LustreError: 228745:0:(osp_precreate.c:1268:osp_precreate_thread()) cfs_fail_timeout id 2107 sleeping for 20000ms [ 7251.427557] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 7251.433088] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7251.451627] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7252.517086] Lustre: lustre-OST0000: Not available for connect from 192.168.202.48@tcp (stopping) [ 7253.486438] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7254.682065] Lustre: server umount lustre-OST0000 complete [ 7266.807426] LustreError: 228745:0:(osp_precreate.c:1268:osp_precreate_thread()) cfs_fail_timeout id 2107 awake [ 7290.709568] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7291.631425] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7293.701582] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7293.734158] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 7302.320213] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 7302.960352] LustreError: lustre-OST0000-osc-MDT0001: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7302.986863] Lustre: lustre-OST0000-osc-MDT0001: Connection restored to 0@lo (at 0@lo) [ 7313.093937] Lustre: DEBUG MARKER: oleg248-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 7315.142983] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid in IDLE state after 0 sec [ 7322.075942] Lustre: DEBUG MARKER: oleg248-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff9206c9dc7000.ost_server_uuid 50 [ 7323.903272] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff9206c9dc7000.ost_server_uuid in IDLE state after 0 sec [ 7333.857539] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7333.873170] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7333.889602] Lustre: Skipped 1 previous similar message [ 7333.893317] Lustre: Skipped 2 previous similar messages [ 7336.132740] Lustre: server umount lustre-OST0000 complete [ 7344.096529] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7344.135066] Lustre: Skipped 1 previous similar message [ 7344.143755] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7344.155326] Lustre: Skipped 2 previous similar messages [ 7349.220860] LustreError: 228725:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7349.242153] LustreError: 228725:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 7349.901607] Lustre: server umount lustre-MDT0000 complete [ 7354.338102] LustreError: 228726:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7354.374341] LustreError: 228726:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 7354.419924] LustreError: 228707:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788447909 with bad export cookie 1229416071795498304 [ 7354.428503] LustreError: MGC192.168.202.148@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7354.435257] LustreError: 228707:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 7354.958253] Lustre: server umount lustre-MDT0001 complete [ 7367.992146] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing unload_modules_local [ 7371.804382] Key type lgssc unregistered [ 7372.127060] LNet: 233939:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 7372.148153] LNetError: 233939:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 7372.166422] LNet: Removed LNI 192.168.202.148@tcp [ 7373.407422] Key type .llcrypt unregistered [ 7373.411107] Key type ._llcrypt unregistered [ 7409.649120] Key type ._llcrypt registered [ 7409.650458] Key type .llcrypt registered [ 7409.853235] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing load_modules_local [ 7411.547885] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 7411.578410] alg: No test for adler32 (adler32-zlib) [ 7412.585220] Lustre: Lustre: Build Version: 2.17.56_51_g2aa7cea [ 7412.722043] LNet: Added LNI 192.168.202.148@tcp [8/256/0/180] [ 7414.376393] Key type lgssc registered [ 7416.324913] Lustre: Echo OBD driver; http://www.lustre.org/ [ 7428.336842] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 7428.364298] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7429.831965] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7434.728677] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 7439.334714] LustreError: 235675:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7439.383046] LustreError: 235675:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 7444.460361] LustreError: 235674:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7445.237216] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7445.570756] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7450.881640] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 7454.418891] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 7461.991566] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7462.653487] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7468.462923] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 7473.942996] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49728) [ 7478.509675] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7478.818618] Lustre: lustre-OST0001: new disk, initializing [ 7478.822403] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 7478.841221] Lustre: 237984:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 7479.003636] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 7485.914841] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 7487.213766] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 7487.231552] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 7487.433831] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 7497.502443] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7507.385489] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 7527.394633] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7527.402846] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7527.422713] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7528.416779] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7528.418298] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7528.448714] Lustre: Skipped 1 previous similar message [ 7530.214530] Lustre: server umount lustre-MDT0000 complete [ 7533.539352] LustreError: 236127:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7533.576763] LustreError: 236127:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [ 7534.855574] LustreError: 236410:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788448090 with bad export cookie 13926076190807167446 [ 7534.867979] LustreError: MGC192.168.202.148@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7534.877952] LustreError: 236410:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 7535.654061] Lustre: server umount lustre-MDT0001 complete [ 7550.642334] Lustre: server umount lustre-OST0000 complete [ 7554.531937] Lustre: 234935:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1788448093/real 1788448093] req@ffff93371f3bf100 x1875323615545728/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1788448109 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 7554.565548] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7554.584896] Lustre: Skipped 1 previous similar message [ 7555.039152] Lustre: 234937:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1788448094/real 1788448094] req@ffff9337021fce00 x1875323615545984/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1788448110 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 7555.392350] Lustre: server umount lustre-OST0001 complete [ 7580.227712] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing load_modules_local [ 7590.368668] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7590.723561] LustreError: 241092:0:(ldlm_lib.c:1192:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7590.793391] Lustre: mylustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7595.227560] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 7596.003101] LustreError: 241093:0:(ldlm_lib.c:1192:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7605.223282] LustreError: 241092:0:(ldlm_lib.c:1192:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7605.289837] LustreError: 241092:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 7607.397918] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7607.829898] Lustre: mylustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7614.999499] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 7618.798469] Lustre: 242236:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 7628.584550] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7629.008446] Lustre: mylustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7632.041685] LustreError: 242592:0:(ldlm_lib.c:1192:target_handle_connect()) mylustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7632.052964] Lustre: mylustre-OST0000: new connection from mylustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:100 to 0x280000400:129) [ 7635.143122] Lustre: mylustre-OST0000: new connection from mylustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49760) [ 7635.912705] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 7644.858309] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7651.463726] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 7659.886779] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7664.090806] Lustre: 244106:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 7675.877460] Lustre: mylustre-MDT0000-lwp-OST0000: Connection to mylustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7675.888870] Lustre: mylustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7675.900282] Lustre: Skipped 2 previous similar messages [ 7675.928358] Lustre: Skipped 6 previous similar messages [ 7680.993494] Lustre: mylustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7681.024101] Lustre: Skipped 3 previous similar messages [ 7681.666831] Lustre: server umount mylustre-MDT0000 complete [ 7685.421110] LustreError: 241068:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788448240 with bad export cookie 13926076190807170309 [ 7685.425438] LustreError: MGC192.168.202.148@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7685.438955] LustreError: 241068:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 7685.950988] Lustre: server umount mylustre-MDT0001 complete [ 7698.937144] Lustre: server umount mylustre-OST0000 complete [ 7701.990519] Lustre: 234937:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1788448241/real 1788448241] req@ffff93371cef5500 x1875323615610752/t0(0) o400->mylustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1788448257 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 7702.032241] Lustre: mylustre-MDT0001-lwp-OST0001: Connection to mylustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7702.060495] Lustre: Skipped 1 previous similar message [ 7704.231085] Lustre: server umount mylustre-OST0001 complete [ 7727.743113] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing load_modules_local [ 7739.602060] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7740.241844] LustreError: 246415:0:(ldlm_lib.c:1192:target_handle_connect()) tfs-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7740.264973] LustreError: 246415:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 4 previous similar messages [ 7740.349938] Lustre: tfs-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7740.360317] Lustre: Skipped 1 previous similar message [ 7745.313497] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 7754.773329] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7761.273664] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 7765.249797] Lustre: 247559:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 7774.462672] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7780.045708] Lustre: tfs-OST0000: new connection from tfs-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:100 to 0x280000400:161) [ 7783.050406] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 7785.219711] Lustre: tfs-OST0000: new connection from tfs-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49792) [ 7793.069564] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7798.826454] Lustre: tfs-OST0001: new connection from tfs-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:35 to 0x2c0000400:65) [ 7800.521318] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 7809.130568] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7813.127652] Lustre: 249430:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 7821.804958] LustreError: tfs-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7821.818347] Lustre: tfs-MDT0000-osp-MDT0001: Connection to tfs-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7821.832149] Lustre: tfs-MDT0000: Not available for connect from 0@lo (stopping) [ 7827.393264] Lustre: server umount tfs-MDT0000 complete [ 7829.487277] LustreError: 248591:0:(ldlm_lib.c:1192:target_handle_connect()) tfs-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7829.517496] LustreError: 248591:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 10 previous similar messages [ 7831.899313] LustreError: 246391:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788448387 with bad export cookie 13926076190807172255 [ 7831.913748] LustreError: MGC192.168.202.148@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7831.914089] LustreError: 246391:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 7832.518777] Lustre: server umount tfs-MDT0001 complete [ 7847.741475] Lustre: server umount tfs-OST0000 complete [ 7849.962843] Lustre: 234937:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1788448389/real 1788448389] req@ffff9337160e5c00 x1875323615664384/t0(0) o400->tfs-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1788448405 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 7850.010284] Lustre: tfs-MDT0001-lwp-OST0001: Connection to tfs-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7850.034806] Lustre: Skipped 3 previous similar messages [ 7852.255558] Lustre: server umount tfs-OST0001 complete [ 7876.631207] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing load_modules_local [ 7890.524455] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7890.827191] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7890.830447] Lustre: Skipped 3 previous similar messages [ 7898.274733] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 7908.900601] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7914.996371] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 7919.366232] Lustre: 252823:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 7929.466834] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7934.217117] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:100 to 0x280000400:193) [ 7935.246030] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49824) [ 7937.327756] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 7946.327140] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7951.878822] Lustre: lustre-OST0001: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:35 to 0x2c0000400:97) [ 7953.241679] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 7961.884902] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7965.795777] Lustre: 254693:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 7972.920385] Lustre: DEBUG MARKER: == conf-sanity test 104a: Make sure user defined options are reflected in mount ========================================================== 11:15:26 (1788448526) [ 7991.284116] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7991.291119] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7991.312929] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7991.326908] Lustre: Skipped 4 previous similar messages [ 7996.490713] Lustre: server umount lustre-MDT0000 complete [ 7997.931481] LustreError: 251681:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7997.975298] LustreError: 251681:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 11 previous similar messages [ 8001.193910] LustreError: 251656:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788448556 with bad export cookie 13926076190807174180 [ 8001.201174] LustreError: MGC192.168.202.148@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8001.211893] LustreError: 251656:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 8007.843425] Lustre: server umount lustre-MDT0001 complete [ 8027.105042] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 8027.307360] Lustre: server umount lustre-OST0000 complete [ 8038.488820] Lustre: server umount lustre-OST0001 complete [ 8046.251459] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_hostid [ 8057.426039] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing load_modules_local [ 8098.468488] LDISKFS-fs (dm-0): Mount option "noacl" will be removed by 3.5 [ 8098.468488] Contact linux-ldiskfs@vger.kernel.org if you think we should keep it. [ 8098.468488] [ 8098.510897] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: acl,user_xattr,errors=remount-ro,noacl,no_mbcache,nodelalloc [ 8098.903962] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 8098.974370] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 8099.116091] Lustre: lustre-MDT0000: new disk, initializing [ 8099.237726] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8099.240609] Lustre: Skipped 3 previous similar messages [ 8099.295833] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 8103.855519] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 8114.968718] LDISKFS-fs (dm-1): Mount option "noacl" will be removed by 3.5 [ 8114.968718] Contact linux-ldiskfs@vger.kernel.org if you think we should keep it. [ 8114.968718] [ 8115.056443] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: acl,user_xattr,errors=remount-ro,noacl,no_mbcache,nodelalloc [ 8115.185278] Lustre: 257884:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 8115.236222] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 8115.240364] Lustre: Skipped 1 previous similar message [ 8115.334295] Lustre: lustre-MDT0001: new disk, initializing [ 8115.541307] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 8115.564967] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 8121.091948] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 8132.348127] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8132.697660] Lustre: lustre-OST0000: new disk, initializing [ 8132.701720] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 8132.706863] Lustre: 259464:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8133.840715] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 8133.854745] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 8133.980583] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 8140.144435] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 8152.343090] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8152.594904] Lustre: lustre-OST0001: new disk, initializing [ 8152.600842] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 8152.606527] Lustre: 260337:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8154.099817] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 8154.116920] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 8154.248507] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 8159.874302] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 8171.784821] Lustre: DEBUG MARKER: == conf-sanity test 104b: Mount uses last flock argument ========================================================== 11:18:46 (1788448726) [ 8188.848416] Lustre: DEBUG MARKER: == conf-sanity test 105: check file creation for ro and rw bind mnt pt ========================================================== 11:19:03 (1788448743) [ 8205.281846] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8205.295040] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8205.324310] Lustre: Skipped 6 previous similar messages [ 8205.347342] Lustre: Skipped 9 previous similar messages [ 8210.594990] Lustre: server umount lustre-OST0000 complete [ 8220.976729] Lustre: server umount lustre-MDT0000 complete [ 8224.317363] LustreError: 260336:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788448779 with bad export cookie 13926076190807175958 [ 8224.319645] LustreError: MGC192.168.202.148@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8224.332304] LustreError: 260336:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 8230.637548] Lustre: server umount lustre-MDT0001 complete [ 8241.023401] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing unload_modules_local [ 8313.541182] Lustre: server umount lustre-OST0001 complete [ 8322.341680] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_hostid [ 8338.756835] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing load_modules_local [ 8375.027978] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8375.313756] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 8375.350309] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 8375.419257] Lustre: lustre-MDT0000: new disk, initializing [ 8375.504439] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8375.521466] Lustre: Skipped 3 previous similar messages [ 8375.569952] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 8380.892867] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 8391.255861] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8391.375685] Lustre: 265135:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 8391.424893] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 8391.428466] Lustre: Skipped 1 previous similar message [ 8391.484267] Lustre: lustre-MDT0001: new disk, initializing [ 8391.616929] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 8391.632619] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 8396.710221] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 8406.285848] Lustre: DEBUG MARKER: oleg248-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 8411.636986] Lustre: DEBUG MARKER: oleg248-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 8418.051156] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8418.320925] Lustre: lustre-OST0000: new disk, initializing [ 8418.325274] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 8418.331775] Lustre: 267102:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8420.273125] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 8420.277878] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 8420.414305] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 8424.759823] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 8433.303524] Lustre: DEBUG MARKER: oleg248-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8440.806032] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8440.806947] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 8440.815066] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8440.815088] Lustre: Skipped 10 previous similar messages [ 8440.819046] Lustre: Skipped 5 previous similar messages [ 8445.759893] Lustre: server umount lustre-OST0000 complete [ 8455.681900] Lustre: server umount lustre-MDT0000 complete [ 8456.166064] LustreError: 265142:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8456.166064] LustreError: 265146:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8456.166077] LustreError: 265142:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 8 previous similar messages [ 8459.789475] LustreError: 265126:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788449014 with bad export cookie 13926076190807177428 [ 8459.794870] LustreError: MGC192.168.202.148@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8459.816698] LustreError: 265126:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 8460.227152] Lustre: server umount lustre-MDT0001 complete [ 8470.361792] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing unload_modules_local [ 8472.979500] Key type lgssc unregistered [ 8473.220864] LNet: 268867:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8473.227587] LNetError: 268867:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8473.244954] LNet: Removed LNI 192.168.202.148@tcp [ 8474.047155] Key type .llcrypt unregistered [ 8474.049069] Key type ._llcrypt unregistered [ 8499.919622] Key type ._llcrypt registered [ 8499.921966] Key type .llcrypt registered [ 8500.040794] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing load_modules_local [ 8501.304677] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8501.325671] alg: No test for adler32 (adler32-zlib) [ 8502.410975] Lustre: Lustre: Build Version: 2.17.56_51_g2aa7cea [ 8502.693587] LNet: Added LNI 192.168.202.148@tcp [8/256/0/180] [ 8504.367180] Key type lgssc registered [ 8505.677742] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8513.329912] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 8513.350545] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8514.729722] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8518.987688] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 8520.164643] LustreError: 270327:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8520.175349] LustreError: 270327:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 8524.271989] LustreError: 270328:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8526.867165] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8527.189672] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8531.977543] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 8540.179296] Lustre: DEBUG MARKER: oleg248-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 8546.380645] Lustre: DEBUG MARKER: oleg248-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 8552.242651] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8552.495248] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8558.360910] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 8559.673686] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:3 to 0x280000401:33) [ 8567.969911] Lustre: DEBUG MARKER: oleg248-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8569.582680] Lustre: Setting parameter lustre-OST0000.ost.unknown_param=50 in log lustre-OST0000 [ 8574.945647] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 8574.955221] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8574.968774] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8576.993167] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 8577.000963] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8577.008242] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8578.480192] Lustre: server umount lustre-OST0000 complete [ 8583.648387] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8583.654753] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8583.666716] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8583.672987] Lustre: Skipped 1 previous similar message [ 8587.243958] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8587.262911] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8589.556735] Lustre: server umount lustre-MDT0000 complete [ 8592.363739] LustreError: 271057:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8592.397114] LustreError: 271057:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 8594.532341] LustreError: 272858:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788449149 with bad export cookie 7572298858879152585 [ 8594.534109] LustreError: MGC192.168.202.148@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8594.535891] LustreError: 272858:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 8594.945772] Lustre: server umount lustre-MDT0001 complete [ 8607.706560] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing unload_modules_local [ 8611.511759] Key type lgssc unregistered [ 8611.888824] LNet: 273918:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8611.898772] LNetError: 273918:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8611.930329] LNet: Removed LNI 192.168.202.148@tcp [ 8612.880695] Key type .llcrypt unregistered [ 8612.884067] Key type ._llcrypt unregistered [ 8631.807703] Key type ._llcrypt registered [ 8631.816147] Key type .llcrypt registered [ 8631.918793] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing load_modules_local [ 8633.352045] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8633.438870] alg: No test for adler32 (adler32-zlib) [ 8634.496860] Lustre: Lustre: Build Version: 2.17.56_51_g2aa7cea [ 8634.750623] LNet: Added LNI 192.168.202.148@tcp [8/256/0/180] [ 8636.431358] Key type lgssc registered [ 8637.921938] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8647.952360] Lustre: lustre-OST0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 8647.970859] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8663.584427] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 8668.703347] LustreError: 275114:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.202.148@tcp: failed processing log, type 4: rc = -110 [ 8695.391473] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8701.019095] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 8708.434122] Lustre: DEBUG MARKER: oleg248-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8710.635432] Lustre: server umount lustre-OST0000 complete [ 8725.466867] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing unload_modules_local [ 8727.879686] Key type lgssc unregistered [ 8728.189536] LNet: 276749:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8728.196651] LNetError: 276749:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8729.255436] LNet: Removed LNI 192.168.202.148@tcp [ 8729.959206] Key type .llcrypt unregistered [ 8729.961361] Key type ._llcrypt unregistered [ 8771.883412] Key type ._llcrypt registered [ 8771.884843] Key type .llcrypt registered [ 8771.949775] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing load_modules_local [ 8773.057326] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8773.216500] alg: No test for adler32 (adler32-zlib) [ 8774.521569] Lustre: Lustre: Build Version: 2.17.56_51_g2aa7cea [ 8774.798029] LNet: Added LNI 192.168.202.148@tcp [8/256/0/180] [ 8776.520294] Key type lgssc registered [ 8777.723404] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8792.392801] loop0: detected capacity change from 0 to 204800000 [ 8792.659757] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8792.717037] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8793.188273] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8794.305418] loop0: detected capacity change from 0 to 204800000 [ 8794.520312] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8794.530902] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8795.017222] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8796.295403] loop0: detected capacity change from 0 to 204800000 [ 8796.527394] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8796.537114] blk_update_request: operation not supported error, dev loop0, sector 8224 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8796.807058] blk_update_request: operation not supported error, dev loop0, sector 58264 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8798.301817] loop0: detected capacity change from 0 to 204800000 [ 8798.550945] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8798.575558] blk_update_request: operation not supported error, dev loop0, sector 8224 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8798.748991] blk_update_request: operation not supported error, dev loop0, sector 58264 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8800.148674] loop0: detected capacity change from 0 to 204800000 [ 8800.187896] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 8804.833945] loop0: detected capacity change from 0 to 204800000 [ 8804.865058] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 8808.885397] loop0: detected capacity change from 0 to 204800000 [ 8808.937039] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 8813.187566] loop0: detected capacity change from 0 to 204800000 [ 8813.234872] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 8817.596675] loop0: detected capacity change from 0 to 204800000 [ 8817.759347] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 8817.840760] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8817.938375] Lustre: 280164:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 8817.949583] Lustre: 280164:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index 'nodemap' with [0x200000003:0x3:0x0]: rc = 0 [ 8817.957711] Lustre: 280164:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000-MDT0000' with [0x200000005:0x3:0x0]: rc = 0 [ 8817.969669] Lustre: 280164:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000003:0xb:0x0]: rc = 0 [ 8817.982582] Lustre: 280164:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000003:0xc:0x0]: rc = 0 [ 8817.992803] Lustre: 280164:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000-MDT0000' with [0x200000005:0x2:0x0]: rc = 0 [ 8818.006909] Lustre: 280164:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000-MDT0000' with [0x200000005:0x1:0x0]: rc = 0 [ 8818.021916] Lustre: 280164:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000003:0xa:0x0]: rc = 0 [ 8818.036812] Lustre: 280164:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000006:0x1010000:0x0]: rc = 0 [ 8818.055264] Lustre: 280164:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000006:0x2010000:0x0]: rc = 0 [ 8818.069686] Lustre: 280164:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000006:0x10000:0x0]: rc = 0 [ 8818.082887] Lustre: 280164:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x20000' with [0x200000006:0x20000:0x0]: rc = 0 [ 8818.107520] Lustre: 280164:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2020000' with [0x200000006:0x2020000:0x0]: rc = 0 [ 8818.126988] Lustre: 280164:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1020000' with [0x200000006:0x1020000:0x0]: rc = 0 [ 8823.765994] Lustre: server umount MGS complete [ 8824.571639] loop0: detected capacity change from 0 to 204800000 [ 8824.712876] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8825.308361] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8825.374946] Lustre: lustre-MDT0000: Aborting client recovery [ 8825.422524] LustreError: 280542:0:(lod_dev.c:511:lod_sub_recovery_thread()) lustre-MDT0000-osd: get update log duration 0, retries 0, failed: rc = -108 [ 8826.276799] loop1: detected capacity change from 0 to 204800000 [ 8826.346140] LDISKFS-fs (loop1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8826.418302] Lustre: 280599:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 8826.428348] Lustre: 280599:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x10000-MDT0001' with [0x200000005:0x1:0x0]: rc = 0 [ 8826.438298] Lustre: 280599:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x10000' with [0x200000003:0x4:0x0]: rc = 0 [ 8826.448849] Lustre: 280599:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x1010000' with [0x200000003:0x5:0x0]: rc = 0 [ 8826.455136] Lustre: 280599:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x1010000-MDT0001' with [0x200000005:0x2:0x0]: rc = 0 [ 8826.464429] Lustre: 280599:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x2010000' with [0x200000003:0x6:0x0]: rc = 0 [ 8826.475353] Lustre: 280599:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x2010000-MDT0001' with [0x200000005:0x3:0x0]: rc = 0 [ 8826.518064] Lustre: Found index 1 for lustre-MDT0001, updating log [ 8826.787679] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8826.823966] Lustre: lustre-MDT0001: Aborting client recovery [ 8827.897800] loop2: detected capacity change from 0 to 204800000 [ 8828.078135] LDISKFS-fs (loop2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8828.190140] Lustre: 280677:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 8828.211778] Lustre: 280677:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 8828.226107] Lustre: 280677:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x20000-OST0000' with [0x200000005:0x1:0x0]: rc = 0 [ 8828.248090] Lustre: 280677:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x2020000-OST0000' with [0x200000005:0x3:0x0]: rc = 0 [ 8828.257710] Lustre: 280677:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 8828.273723] Lustre: 280677:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x1020000-OST0000' with [0x200000005:0x2:0x0]: rc = 0 [ 8828.286305] Lustre: 280677:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 8828.374695] Lustre: Found index 0 for lustre-OST0000, updating log [ 8828.429546] Lustre: 280695:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8828.497153] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8828.512190] Lustre: lustre-OST0000: Aborting recovery [ 8829.340256] loop3: detected capacity change from 0 to 204800000 [ 8829.441184] LDISKFS-fs (loop3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8829.515717] Lustre: 280760:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 8829.526140] Lustre: 280760:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 8829.544777] Lustre: 280760:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x1020000-OST0001' with [0x200000005:0x2:0x0]: rc = 0 [ 8829.587850] Lustre: 280760:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 8829.631741] Lustre: 280760:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x20000-OST0001' with [0x200000005:0x1:0x0]: rc = 0 [ 8829.659226] Lustre: 280760:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 8829.684469] Lustre: 280760:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x2020000-OST0001' with [0x200000005:0x3:0x0]: rc = 0 [ 8829.717713] Lustre: Found index 1 for lustre-OST0001, updating log [ 8829.737490] Lustre: 280765:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8829.825365] Lustre: lustre-OST0001: Aborting recovery [ 8834.920155] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 8834.947417] LustreError: 280543:0:(lod_dev.c:511:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 10, retries 0, failed: rc = -108 [ 8834.956412] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 8834.971550] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000bd0 [ 8835.001079] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000bd0 [ 8835.939648] Lustre: lustre-OST0001: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:5 to 0x2c0000400:33) [ 8835.940324] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:5 to 0x280000400:33) [ 8837.448558] LustreError: 280510:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0000: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0000 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0000/identity_upcall, time 1944us: rc = -2 [ 8837.952221] LustreError: 280508:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0000: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0000 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0000/identity_upcall, time 1063us: rc = -2 [ 8837.980673] LustreError: 280508:0:(mdt_identity.c:115:mdt_identity_do_upcall()) Skipped 24 previous similar messages [ 8851.425146] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8851.426289] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8851.437863] Lustre: Skipped 2 previous similar messages [ 8851.467995] Lustre: Skipped 3 previous similar messages [ 8856.546254] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8856.558643] Lustre: Skipped 3 previous similar messages [ 8857.021150] Lustre: server umount lustre-MDT0000 complete [ 8858.366344] LustreError: 280493:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788449413 with bad export cookie 6578930699684980433 [ 8858.377123] LustreError: MGC192.168.202.148@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8858.396267] LustreError: 280493:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 8861.679079] LustreError: 281045:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8861.687466] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8861.694841] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 8861.706765] LustreError: 281045:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [ 8861.746386] Lustre: Skipped 2 previous similar messages [ 8863.712117] LustreError: 281026:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8863.714561] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 8863.747506] LustreError: 281026:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 8863.775752] Lustre: Skipped 2 previous similar messages [ 8865.058380] Lustre: server umount lustre-MDT0001 complete [ 8872.921345] Lustre: server umount lustre-OST0000 complete [ 8888.288093] Lustre: lustre-OST0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 8888.719344] Lustre: server umount lustre-OST0001 complete [ 8898.583403] Lustre: DEBUG MARKER: == conf-sanity test 109a: test lctl clear_conf fsname ==== 11:30:52 (1788449452) [ 8930.773739] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_hostid [ 8942.731806] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing load_modules_local [ 8983.073817] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8983.330696] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 8983.365796] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 8983.446633] Lustre: lustre-MDT0000: new disk, initializing [ 8983.559426] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8983.565894] Lustre: Skipped 1 previous similar message [ 8983.588529] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 8989.586312] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 9003.124777] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9003.339398] Lustre: 284832:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 9003.472556] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 9003.481015] Lustre: Skipped 1 previous similar message [ 9003.634953] Lustre: lustre-MDT0001: new disk, initializing [ 9003.828777] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9003.898135] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 9003.909751] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 9010.469568] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 9020.532549] Lustre: DEBUG MARKER: oleg248-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 9026.966925] Lustre: DEBUG MARKER: oleg248-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 9034.315216] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9034.619394] Lustre: lustre-OST0000: new disk, initializing [ 9034.628696] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 9034.635933] Lustre: 286802:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9034.775947] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9041.521055] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 9041.537057] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 9041.732597] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 9041.988882] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 9053.120511] Lustre: DEBUG MARKER: oleg248-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9058.679808] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [ 9069.421036] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [ 9077.706777] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [ 9128.930616] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9128.933398] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 9128.958686] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9128.970875] Lustre: Skipped 1 previous similar message [ 9133.021970] Lustre: server umount lustre-OST0000 complete [ 9139.168809] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9139.179398] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9139.195797] Lustre: Skipped 2 previous similar messages [ 9139.224579] Lustre: Skipped 1 previous similar message [ 9144.288839] LustreError: 284842:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9144.301681] LustreError: 284842:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 9146.019760] Lustre: server umount lustre-MDT0000 complete [ 9149.410925] LustreError: 284845:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9149.429512] LustreError: 284845:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 9150.067580] LustreError: 284825:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788449705 with bad export cookie 6578930699684983142 [ 9150.078903] LustreError: MGC192.168.202.148@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9150.081669] LustreError: 284825:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 9150.740150] Lustre: server umount lustre-MDT0001 complete [ 9161.081780] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9166.683446] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 9172.151996] Lustre: server umount MGS complete [ 9186.370503] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9186.778391] LustreError: 290673:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9186.808829] LustreError: 290686:0:(qmt_pool.c:1639:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [ 9186.878070] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9191.315853] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 9196.004600] LustreError: 290673:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9196.031157] LustreError: 290673:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 9199.458595] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9204.104724] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 9211.903959] Lustre: DEBUG MARKER: oleg248-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 9219.779937] Lustre: DEBUG MARKER: oleg248-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 9227.073950] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9227.535257] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9227.549966] Lustre: Skipped 1 previous similar message [ 9235.496830] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 9243.870403] Lustre: DEBUG MARKER: oleg248-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9275.889196] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9275.891858] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 9275.914916] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9275.919104] LustreError: Skipped 1 previous similar message [ 9275.931690] Lustre: Skipped 1 previous similar message [ 9279.234587] Lustre: server umount lustre-OST0000 complete [ 9286.112778] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9286.123357] Lustre: Skipped 1 previous similar message [ 9289.456539] Lustre: server umount lustre-MDT0000 complete [ 9291.235742] LustreError: 290673:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9291.259846] LustreError: 290673:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 9293.639088] LustreError: 290653:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788449848 with bad export cookie 6578930699684984983 [ 9293.641082] LustreError: MGC192.168.202.148@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9293.647742] LustreError: 290653:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 9294.059864] Lustre: server umount lustre-MDT0001 complete [ 9305.702606] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing unload_modules_local [ 9309.124866] Key type lgssc unregistered [ 9309.506460] LNet: 294622:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 9309.514701] LNetError: 294622:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 9310.567832] LNet: Removed LNI 192.168.202.148@tcp [ 9311.503294] Key type .llcrypt unregistered [ 9311.507720] Key type ._llcrypt unregistered [ 9357.634653] Key type ._llcrypt registered [ 9357.636622] Key type .llcrypt registered [ 9357.790926] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_hostid [ 9373.324338] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing load_modules_local [ 9374.989313] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 9375.252360] alg: No test for adler32 (adler32-zlib) [ 9376.289617] Lustre: Lustre: Build Version: 2.17.56_51_g2aa7cea [ 9376.515532] LNet: Added LNI 192.168.202.148@tcp [8/256/0/180] [ 9378.151231] Key type lgssc registered [ 9380.183823] Lustre: Echo OBD driver; http://www.lustre.org/ [ 9417.363681] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 9417.396266] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9418.753933] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 9418.786084] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 9418.895957] Lustre: lustre-MDT0000: new disk, initializing [ 9418.963296] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9418.981265] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 9423.399711] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 9434.823752] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9434.959995] Lustre: 298582:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 9435.005313] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 9435.008842] Lustre: Skipped 1 previous similar message [ 9435.113614] Lustre: lustre-MDT0001: new disk, initializing [ 9435.252411] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9435.302819] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 9435.318696] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 9439.974317] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 9450.437394] Lustre: DEBUG MARKER: oleg248-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 9457.237232] Lustre: DEBUG MARKER: oleg248-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 9464.626041] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9464.965601] Lustre: lustre-OST0000: new disk, initializing [ 9464.969674] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 9464.976073] Lustre: 300553:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9465.128875] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9471.612378] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 9471.645570] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 9471.796407] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 9472.055918] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 9481.829702] Lustre: DEBUG MARKER: oleg248-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9488.070902] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [ 9499.097712] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [ 9504.618839] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [ 9510.111507] Lustre: Modifying parameter lustre-client.llite.max_read_ahead_mb=64 in log lustre-client [ 9559.027296] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9559.035790] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9559.043741] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 9559.052033] Lustre: Skipped 1 previous similar message [ 9559.081704] Lustre: Skipped 1 previous similar message [ 9563.707871] Lustre: server umount lustre-OST0000 complete [ 9568.229080] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 9568.238685] LustreError: Skipped 1 previous similar message [ 9568.246498] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9568.268027] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9569.251185] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9574.030555] Lustre: server umount lustre-MDT0000 complete [ 9577.799805] LustreError: 298576:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788450132 with bad export cookie 43042325024049653 [ 9577.810413] LustreError: MGC192.168.202.148@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9577.821718] LustreError: 298576:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 9578.242665] Lustre: server umount lustre-MDT0001 complete [ 9587.584611] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9593.040845] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 9598.255321] Lustre: server umount MGS complete [ 9609.197413] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9609.709151] LustreError: 304442:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9609.734485] LustreError: 304442:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [ 9609.760479] LustreError: 304455:0:(qmt_pool.c:1639:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [ 9609.828615] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9614.818874] LustreError: 304443:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9615.282364] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 9618.913878] LustreError: 304442:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9624.043959] LustreError: 304443:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9624.429183] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9624.921867] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9631.043286] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 9640.182706] Lustre: DEBUG MARKER: oleg248-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 9646.304496] Lustre: DEBUG MARKER: oleg248-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 9653.218291] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9653.621861] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9660.095679] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 9667.390036] Lustre: DEBUG MARKER: oleg248-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9690.602239] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9690.603506] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 9690.612777] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9690.612786] Lustre: Skipped 2 previous similar messages [ 9690.617450] Lustre: Skipped 1 previous similar message [ 9690.648218] LustreError: Skipped 1 previous similar message [ 9695.714031] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9695.721556] Lustre: Skipped 1 previous similar message [ 9696.790299] Lustre: server umount lustre-OST0000 complete [ 9700.842348] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9700.855307] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9700.866555] Lustre: Skipped 1 previous similar message [ 9700.880458] Lustre: Skipped 1 previous similar message [ 9705.957411] LustreError: 304442:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9705.970961] LustreError: 304442:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 9706.040195] Lustre: server umount lustre-MDT0000 complete [ 9709.176342] LustreError: 307280:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788450264 with bad export cookie 43042325024051578 [ 9709.177052] LustreError: MGC192.168.202.148@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9709.194454] LustreError: 307280:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 9709.501761] Lustre: server umount lustre-MDT0001 complete [ 9719.798337] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing unload_modules_local [ 9722.631786] Key type lgssc unregistered [ 9722.862305] LNet: 308393:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 9722.870399] LNetError: 308393:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 9722.894792] LNet: Removed LNI 192.168.202.148@tcp [ 9723.803364] Key type .llcrypt unregistered [ 9723.806827] Key type ._llcrypt unregistered [ 9741.256456] Key type ._llcrypt registered [ 9741.258873] Key type .llcrypt registered [ 9741.339186] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_hostid [ 9754.045930] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing load_modules_local [ 9755.388938] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 9755.432135] alg: No test for adler32 (adler32-zlib) [ 9756.504114] Lustre: Lustre: Build Version: 2.17.56_51_g2aa7cea [ 9756.807805] LNet: Added LNI 192.168.202.148@tcp [8/256/0/180] [ 9758.503194] Key type lgssc registered [ 9759.673575] Lustre: Echo OBD driver; http://www.lustre.org/ [ 9781.390345] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing load_modules_local [ 9797.611354] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 9797.639187] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9798.812267] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 9798.829826] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 9798.875838] Lustre: lustre-MDT0000: new disk, initializing [ 9798.922595] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9798.940170] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 9802.945892] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 9812.148181] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9812.437550] Lustre: lustre-OST0000: new disk, initializing [ 9812.445179] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 9812.448990] Lustre: Skipped 1 previous similar message [ 9812.453636] Lustre: 312420:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9812.522519] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9818.317928] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [ 9820.729354] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 9820.734908] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 9820.771294] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [10642.922834] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10642.942602] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10644.507667] Lustre: server umount lustre-MDT0000 complete [10649.327146] LustreError: 311607:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788451204 with bad export cookie 18284036239367274059 [10649.334618] LustreError: MGC192.168.202.148@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10649.342142] LustreError: 311607:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [10649.480737] Lustre: server umount lustre-OST0000 complete [10675.138383] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_hostid [10684.349769] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing load_modules_local [10722.308746] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10722.574238] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [10722.621090] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [10722.766597] Lustre: lustre-MDT0000: new disk, initializing [10722.920228] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10722.951885] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [10728.431665] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [10740.095337] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10740.277191] Lustre: 316772:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [10740.346607] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [10740.350478] Lustre: Skipped 1 previous similar message [10740.487328] Lustre: lustre-MDT0001: new disk, initializing [10740.583515] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10740.625156] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [10740.642972] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [10746.467380] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [10757.530335] Lustre: DEBUG MARKER: oleg248-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [10764.610770] Lustre: DEBUG MARKER: oleg248-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [10772.400637] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10772.663832] Lustre: lustre-OST0000: new disk, initializing [10772.667074] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [10772.673989] Lustre: 318741:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [10772.763139] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10779.721807] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [10779.731963] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [10779.848037] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [10780.008613] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [10789.677583] Lustre: DEBUG MARKER: oleg248-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [10794.888545] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [10795.122709] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [10799.972220] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [10800.169654] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [10805.732762] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10805.737349] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [10805.748925] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10805.761184] LustreError: Skipped 1 previous similar message [10807.922639] Lustre: server umount lustre-OST0000 complete [10812.389719] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [10812.415884] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10812.439818] Lustre: Skipped 1 previous similar message [10812.447159] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10812.449718] Lustre: Skipped 1 previous similar message [10815.972890] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10815.979594] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10818.413625] Lustre: server umount lustre-MDT0000 complete [10822.996603] LustreError: 316764:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788451378 with bad export cookie 18284036239370214878 [10823.000949] LustreError: MGC192.168.202.148@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10823.010614] LustreError: 316764:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [10823.265938] Lustre: server umount lustre-MDT0001 complete [10827.368559] Lustre: DEBUG MARKER: SKIP: conf-sanity test_111 skipping SLOW test 111 [10830.124913] Lustre: DEBUG MARKER: == conf-sanity test 112a: mount OST with no_create option ========================================================== 12:03:03 (1788451383) [10837.453195] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10837.836176] LustreError: 321055:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10837.852793] LustreError: 321055:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [10837.899558] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10842.438936] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [10843.109346] LustreError: 321056:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10848.229238] LustreError: 321055:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10850.746182] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10851.139282] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10855.977798] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [10864.600419] Lustre: DEBUG MARKER: oleg248-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [10871.875269] Lustre: DEBUG MARKER: oleg248-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [10879.274582] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10879.727062] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10887.382916] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [10895.251432] Lustre: DEBUG MARKER: oleg248-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [10903.276703] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10903.575298] Lustre: lustre-OST0001: new disk, initializing [10903.584438] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [10903.605847] Lustre: 323893:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [10903.749214] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [10911.167362] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [10911.865525] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [10911.883327] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [10911.957678] LustreError: 324279:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [10913.004392] LustreError: 324279:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [10913.024555] LustreError: 324279:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 1 previous similar message [10914.120632] LustreError: 324287:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [10914.146432] LustreError: 324287:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 2 previous similar messages [10916.192845] LustreError: 324279:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [10916.214685] LustreError: 324279:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 2 previous similar messages [10920.433057] LustreError: 324287:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [10920.450558] LustreError: 324287:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 8 previous similar messages [10922.825304] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [10923.058466] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [10928.677439] LustreError: 324279:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [10928.689479] LustreError: 324279:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 14 previous similar messages [10930.661361] Lustre: DEBUG MARKER: oleg248-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff9206cabb1800.ost_server_uuid 50 [10933.524432] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff9206cabb1800.ost_server_uuid in FULL state after 0 sec [10939.685527] Lustre: DEBUG MARKER: oleg248-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff9206cabb1800.ost_server_uuid 50 [10941.323902] Lustre: DEBUG MARKER: osc.lustre-OST0001-osc-ffff9206cabb1800.ost_server_uuid in IDLE state after 0 sec [10943.154860] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [10958.315548] Lustre: lustre-OST0001-osc-MDT0000: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10958.346119] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [10958.362407] Lustre: Skipped 2 previous similar messages [10963.069470] Lustre: server umount lustre-OST0001 complete [10963.426817] LustreError: 322875:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10963.462591] LustreError: 322875:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [10968.582796] LustreError: 322875:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10968.616636] LustreError: 322875:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [10971.114625] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [10971.130072] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10971.151822] Lustre: Skipped 1 previous similar message [10971.167030] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10971.173698] Lustre: Skipped 1 previous similar message [10973.154937] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [10975.648337] Lustre: server umount lustre-OST0000 complete [10987.033966] Lustre: server umount lustre-MDT0000 complete [10989.025100] LustreError: 322457:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10989.058508] LustreError: 322457:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [10992.193630] LustreError: 321036:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788451547 with bad export cookie 18284036239370215760 [10992.203455] LustreError: MGC192.168.202.148@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10992.208161] LustreError: 321036:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [10992.781944] Lustre: server umount lustre-MDT0001 complete [11006.894419] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing unload_modules_local [11009.608179] Key type lgssc unregistered [11009.953228] LNet: 326585:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [11009.958437] LNetError: 326585:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [11010.989811] LNet: Removed LNI 192.168.202.148@tcp [11012.197386] Key type .llcrypt unregistered [11012.204901] Key type ._llcrypt unregistered [11040.906385] Key type ._llcrypt registered [11040.909662] Key type .llcrypt registered [11041.113745] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing load_modules_local [11042.857568] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [11042.889848] alg: No test for adler32 (adler32-zlib) [11044.124713] Lustre: Lustre: Build Version: 2.17.56_51_g2aa7cea [11044.489839] LNet: Added LNI 192.168.202.148@tcp [8/256/0/180] [11046.192058] Key type lgssc registered [11047.817970] Lustre: Echo OBD driver; http://www.lustre.org/ [11056.626811] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [11056.647420] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11058.249259] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11063.871842] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [11068.392591] LustreError: 328008:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11068.419816] LustreError: 328008:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [11072.108347] Lustre: DEBUG MARKER: oleg248-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [11073.509748] LustreError: 328009:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11078.626482] LustreError: 328008:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11079.173204] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11079.494341] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11083.754092] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [11094.620308] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11094.955154] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [11103.166922] LustreError: 329686:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11103.213340] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:65) [11104.708959] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [11109.357895] LustreError: 329689:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11109.388322] LustreError: 329689:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [11115.696676] Lustre: DEBUG MARKER: oleg248-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [11119.587941] LustreError: 329993:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11119.608886] LustreError: 329993:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [11123.588250] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11123.992122] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [11129.380278] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:33) [11131.459342] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [11139.797147] Lustre: DEBUG MARKER: oleg248-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [11147.808925] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing wait_import_state (FULL|IDLE) os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [11148.140665] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [11252.193855] LustreError: lustre-OST0001-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [11252.196924] Lustre: lustre-OST0001-osc-MDT0001: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11252.198380] LustreError: Skipped 1 previous similar message [11252.222144] Lustre: Skipped 1 previous similar message [11252.230730] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [11256.699618] Lustre: server umount lustre-OST0001 complete [11257.331768] LustreError: 329689:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11257.344252] LustreError: 329689:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [11267.042092] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [11267.055792] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11267.075347] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [11267.092356] Lustre: Skipped 1 previous similar message [11272.674974] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [11272.701740] Lustre: Skipped 2 previous similar messages [11277.282548] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [11277.604898] Lustre: server umount lustre-OST0000 complete [11282.913465] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11282.932453] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11282.952052] Lustre: Skipped 2 previous similar messages [11288.033842] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11288.048842] Lustre: Skipped 2 previous similar messages [11288.434702] Lustre: server umount lustre-MDT0000 complete [11292.786933] LustreError: 327988:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788451847 with bad export cookie 14953800436100602459 [11292.802639] LustreError: MGC192.168.202.148@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11292.811471] LustreError: 327988:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [11293.480635] Lustre: server umount lustre-MDT0001 complete [11308.065886] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing unload_modules_local [11311.386499] Key type lgssc unregistered [11311.770721] LNet: 332962:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [11311.779207] LNetError: 332962:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [11311.794487] LNet: Removed LNI 192.168.202.148@tcp [11312.840409] Key type .llcrypt unregistered [11312.845297] Key type ._llcrypt unregistered [11363.094520] Key type ._llcrypt registered [11363.098286] Key type .llcrypt registered [11363.312722] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing load_modules_local [11364.637823] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [11364.825542] alg: No test for adler32 (adler32-zlib) [11366.158841] Lustre: Lustre: Build Version: 2.17.56_51_g2aa7cea [11366.495797] LNet: Added LNI 192.168.202.148@tcp [8/256/0/180] [11368.191223] Key type lgssc registered [11369.476447] Lustre: Echo OBD driver; http://www.lustre.org/ [11378.792926] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [11378.829472] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11380.369788] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11385.886701] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [11389.927749] LustreError: 335253:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11389.947716] LustreError: 335253:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [11395.042765] LustreError: 335254:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11395.051470] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11395.331435] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11400.659071] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [11403.809637] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [11411.297504] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11411.973355] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [11419.941700] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [11420.143061] LustreError: 336741:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11425.258537] LustreError: 336741:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11425.303483] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:97) [11425.306415] LustreError: 336741:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [11429.486115] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11429.783386] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [11435.026177] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:65) [11436.639884] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [11446.664471] Lustre: DEBUG MARKER: Using TIMEOUT=20 [11454.792390] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [11467.234776] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [11467.245499] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11467.267196] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11470.827782] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11470.829614] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11470.841772] Lustre: Skipped 1 previous similar message [11470.868166] Lustre: Skipped 1 previous similar message [11472.444457] Lustre: server umount lustre-MDT0000 complete [11475.682876] LustreError: 335233:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788452030 with bad export cookie 18160046940144276670 [11475.691039] LustreError: MGC192.168.202.148@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11475.992280] Lustre: server umount lustre-MDT0001 complete [11489.938908] Lustre: server umount lustre-OST0000 complete [11492.831794] Lustre: 334516:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1788452031/real 1788452031] req@ffff93383a513b80 x1875327761579392/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1788452047 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [11492.865784] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11492.879072] Lustre: Skipped 1 previous similar message [11493.989688] Lustre: server umount lustre-OST0001 complete [11511.292331] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing load_modules_local [11522.097336] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11522.786146] LustreError: 340519:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11522.851907] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11527.336754] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [11532.259797] LustreError: 340520:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11532.268950] LustreError: 340520:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [11536.317134] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11536.684964] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11541.267524] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [11543.955632] Lustre: 341661:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [11550.576214] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11556.764275] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [11561.255163] LustreError: 342014:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11561.276754] LustreError: 342014:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [11565.566256] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11565.770582] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [11565.777039] Lustre: Skipped 1 previous similar message [11566.844349] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:129) [11567.933588] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:97) [11572.132303] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [11579.217455] Lustre: DEBUG MARKER: Using TIMEOUT=20 [11582.749231] Lustre: 343529:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [11588.066726] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [11588.079458] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11588.095403] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11588.110224] Lustre: Skipped 2 previous similar messages [11592.888684] Lustre: server umount lustre-MDT0000 complete [11593.702062] LustreError: 341251:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11593.725912] LustreError: 341251:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [11596.027244] LustreError: 340501:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788452151 with bad export cookie 18160046940144278371 [11596.029669] LustreError: MGC192.168.202.148@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11596.042386] LustreError: 340501:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 5 previous similar messages [11596.399797] Lustre: server umount lustre-MDT0001 complete [11610.354479] Lustre: server umount lustre-OST0000 complete [11625.654727] Lustre: server umount lustre-OST0001 complete [11629.761562] Lustre: DEBUG MARKER: SKIP: conf-sanity test_114 skipping SLOW test 114 [11631.248953] Lustre: DEBUG MARKER: SKIP: conf-sanity test_115 skipping excluded test 115 [11633.006632] Lustre: DEBUG MARKER: == conf-sanity test 116: big size MDT support ============ 12:16:27 (1788452187) [11656.537115] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing load_modules_local [11666.160585] loop0: detected capacity change from 0 to 1099511627776 [11666.778339] SGI XFS with ACLs, security attributes, quota, no debug enabled [11666.785535] *** VALIDATE xfs *** [11666.813757] XFS (loop0): Mounting V5 Filesystem [11666.843658] XFS (loop0): Ending clean mount [11668.187578] loop1: detected capacity change from 0 to 18691697672192 [11783.771777] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_hostid [11793.989036] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing load_modules_local [11826.310621] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11826.551553] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [11826.591994] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [11826.699485] Lustre: lustre-MDT0000: new disk, initializing [11826.803602] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11826.827493] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [11830.711750] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [11841.568770] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11841.717810] Lustre: 348666:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [11841.768989] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [11841.772379] Lustre: Skipped 1 previous similar message [11841.881950] Lustre: lustre-MDT0001: new disk, initializing [11842.117717] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [11842.130323] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [11847.383799] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [11856.370298] Lustre: DEBUG MARKER: oleg248-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [11862.959853] Lustre: DEBUG MARKER: oleg248-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [11869.743609] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11870.162639] Lustre: lustre-OST0000: new disk, initializing [11870.165086] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [11870.172563] Lustre: 350634:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [11872.098934] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [11872.114316] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [11872.242945] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [11877.200977] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [11886.214622] Lustre: DEBUG MARKER: oleg248-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [11891.328988] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [11891.494293] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [11895.715400] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [11895.918408] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [11897.842022] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [11897.842791] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11897.847019] LustreError: Skipped 1 previous similar message [11897.850955] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [11897.857825] Lustre: Skipped 4 previous similar messages [11897.861691] Lustre: Skipped 3 previous similar messages [11902.946187] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [11902.954844] Lustre: Skipped 2 previous similar messages [11904.307491] Lustre: server umount lustre-OST0000 complete [11913.185967] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11913.188741] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11913.227484] Lustre: Skipped 1 previous similar message [11913.247613] Lustre: Skipped 1 previous similar message [11915.876448] Lustre: server umount lustre-MDT0000 complete [11918.310268] LustreError: 348672:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11918.337546] LustreError: 348672:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [11920.635675] LustreError: 348656:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788452475 with bad export cookie 18160046940144279925 [11920.648655] LustreError: MGC192.168.202.148@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11920.654976] LustreError: 348656:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [11921.254701] Lustre: server umount lustre-MDT0001 complete [11929.334697] XFS (loop0): Unmounting Filesystem [11934.547738] Lustre: DEBUG MARKER: == conf-sanity test 117: lctl get_param return errors properly ========================================================== 12:21:28 (1788452488) [11941.674808] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11942.288766] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11942.298120] Lustre: Skipped 2 previous similar messages [11947.127447] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [11956.720608] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11961.950323] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [11970.914506] Lustre: DEBUG MARKER: oleg248-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [11977.207862] Lustre: DEBUG MARKER: oleg248-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [11984.239309] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11991.552194] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [11999.885088] Lustre: DEBUG MARKER: oleg248-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [12013.546411] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [12013.548150] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12013.559401] LustreError: Skipped 1 previous similar message [12013.565721] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [12013.598721] Lustre: Skipped 1 previous similar message [12014.896352] Lustre: server umount lustre-OST0000 complete [12023.777506] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [12025.070279] Lustre: server umount lustre-MDT0000 complete [12028.634036] LustreError: 353177:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788452583 with bad export cookie 18160046940144280807 [12028.637596] LustreError: MGC192.168.202.148@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [12028.642513] LustreError: 353177:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [12029.276985] Lustre: server umount lustre-MDT0001 complete [12041.173318] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing unload_modules_local [12043.686382] Key type lgssc unregistered [12043.937722] LNet: 356828:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [12043.949791] LNetError: 356828:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [12043.959903] LNet: Removed LNI 192.168.202.148@tcp [12044.967282] Key type .llcrypt unregistered [12044.971350] Key type ._llcrypt unregistered [12074.472670] Key type ._llcrypt registered [12074.491729] Key type .llcrypt registered [12074.737283] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing load_modules_local [12075.902389] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [12075.945375] alg: No test for adler32 (adler32-zlib) [12077.381789] Lustre: Lustre: Build Version: 2.17.56_51_g2aa7cea [12077.796537] LNet: Added LNI 192.168.202.148@tcp [8/256/0/180] [12079.543650] Key type lgssc registered [12081.555310] Lustre: Echo OBD driver; http://www.lustre.org/ [12092.476387] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [12092.549815] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12094.522492] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [12099.280944] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [12104.681190] LustreError: 358395:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12104.718773] LustreError: 358395:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [12108.232966] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12108.568035] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [12114.430277] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [12124.023793] Lustre: DEBUG MARKER: oleg248-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [12130.318519] Lustre: DEBUG MARKER: oleg248-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [12137.772245] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12138.257205] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [12145.491396] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [12153.044640] Lustre: DEBUG MARKER: oleg248-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [12164.070017] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12164.098847] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [12166.197496] Lustre: lustre-MDT0001: Not available for connect from 192.168.202.48@tcp (stopping) [12166.205215] Lustre: Skipped 1 previous similar message [12169.199332] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [12169.208224] Lustre: Skipped 1 previous similar message [12171.271179] Lustre: lustre-MDT0001: Not available for connect from 192.168.202.48@tcp (stopping) [12176.351134] Lustre: lustre-MDT0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [12176.399393] Lustre: lustre-MDT0001: Not available for connect from 192.168.202.48@tcp (stopping) [12176.428464] Lustre: Skipped 2 previous similar messages [12176.729765] Lustre: server umount lustre-MDT0001 complete [12179.448389] LustreError: 359124:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12179.469094] LustreError: 359124:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [12181.543760] LustreError: 359679:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.202.48@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [12184.557443] LustreError: 358396:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12184.591884] LustreError: 358396:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [12188.637156] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12188.828364] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [12188.837303] Lustre: Found index 1 for lustre-MDT0001, updating log [12188.886931] Lustre: 358382:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [12189.382529] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [12194.798766] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [12194.919547] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [12196.891974] LustreError: 361926:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [12196.898852] LustreError: 361926:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [12196.910794] LustreError: 358387:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [12196.987617] LustreError: 361932:0:(obd_config.c:2103:class_config_llog_handler()) MGC192.168.202.148@tcp: cfg command failed: rc = -17 [12197.006555] Lustre: cmd=cf001 0:lustre-MDT0001-osp-MDT0000 1:osp 2:lustre-MDT0000-mdtlov_UUID [12197.006555] [12207.137991] Lustre: lustre-MDT0001: Not available for connect from 192.168.202.48@tcp (stopping) [12210.151614] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [12210.165300] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12210.176961] Lustre: Skipped 2 previous similar messages [12212.205532] Lustre: server umount lustre-MDT0001 complete [12212.236263] LustreError: 358391:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.202.48@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [12212.263915] LustreError: 358391:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [12220.391514] LustreError: 358395:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12220.425933] LustreError: 358395:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 4 previous similar messages [12223.036533] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12223.197794] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [12223.212967] Lustre: Found index 1 for lustre-MDT0001, updating log [12223.296827] Lustre: 358385:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [12223.829480] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [12228.634774] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [12229.108802] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [12233.220219] LustreError: 363072:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [12233.236496] LustreError: 363072:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [12233.256336] LustreError: 358387:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [12233.266673] LustreError: 358387:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [12233.365650] LustreError: 363075:0:(obd_config.c:761:class_setup()) osp: device 8 already setup: rc = -17 [12233.375786] LustreError: 363075:0:(obd_config.c:866:class_detach()) OBD device 8 still set up [12233.383273] LustreError: 363075:0:(obd_config.c:2103:class_config_llog_handler()) MGC192.168.202.148@tcp: cfg command failed: rc = -17 [12233.395858] Lustre: cmd=cf003 0:lustre-MDT0001-osp-MDT0000 1:lustre-MDT0001_UUID 2:192.168.202.148@tcp [12233.395858] [12534.839500] Lustre: lustre-MDT0001: Not available for connect from 192.168.202.48@tcp (stopping) [12534.846431] Lustre: Skipped 2 previous similar messages [12536.291292] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [12536.291619] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12549.088930] Lustre: lustre-MDT0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [12549.245918] Lustre: server umount lustre-MDT0001 complete [12550.155809] LustreError: 358392:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.202.48@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [12550.179895] LustreError: 358392:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [12559.599658] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12559.714967] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [12559.723822] Lustre: Found index 1 for lustre-MDT0001, updating log [12559.806511] Lustre: 358385:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [12560.125408] Lustre: lustre-MDT0001: Imperative Recovery enabled, recovery window shrunk from 60-180 down to 60-180 [12565.051382] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [12565.514522] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [12568.567608] LustreError: 371546:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [12568.580069] LustreError: 371546:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [12568.593845] LustreError: 358387:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [12568.605116] LustreError: 358387:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [12568.688194] LustreError: 371549:0:(lod_lov.c:210:lod_add_device()) lustre-MDT0001-osp-MDT0000: cannot connect to next dev lustre-MDT0001_UUID (-114) [12568.701450] LustreError: 371549:0:(obd_config.c:2103:class_config_llog_handler()) MGC192.168.202.148@tcp: cfg command failed: rc = -114 [12568.707817] Lustre: cmd=cf014 0:lustre-MDT0000-mdtlov 1:lustre-MDT0001_UUID 2:1 3:1 [12568.707817] [12888.033985] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12888.038040] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [12888.043960] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12888.043969] Lustre: Skipped 8 previous similar messages [12888.063320] Lustre: Skipped 3 previous similar messages [12893.803899] Lustre: server umount lustre-MDT0000 complete [12897.437826] LustreError: 358374:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788453452 with bad export cookie 13169643763217121026 [12897.440849] LustreError: MGC192.168.202.148@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [12897.460170] LustreError: 358374:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [12898.276541] LustreError: 359679:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12898.277555] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12898.297760] LustreError: 359679:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 5 previous similar messages [12903.923697] Lustre: server umount lustre-MDT0001 complete [12907.955732] Lustre: server umount lustre-OST0000 complete [12917.252968] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_hostid [12927.536678] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing load_modules_local [12962.468416] Lustre: DEBUG MARKER: == conf-sanity test 120: cross-target rename should not create bad symlinks ========================================================== 12:38:37 (1788453517) [12968.239063] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12968.437120] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [12968.461022] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [12968.535929] Lustre: lustre-MDT0000: new disk, initializing [12968.608641] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [12968.625934] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [12972.941339] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [12983.829195] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12983.959985] Lustre: 382277:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [12984.011490] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [12984.014901] Lustre: Skipped 1 previous similar message [12984.076960] Lustre: lustre-MDT0001: new disk, initializing [12984.176753] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [12984.222325] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [12984.231334] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [12988.697481] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [12998.474705] Lustre: DEBUG MARKER: oleg248-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [13004.445867] Lustre: DEBUG MARKER: oleg248-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [13010.969955] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [13011.271195] Lustre: lustre-OST0000: new disk, initializing [13011.274413] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [13011.285744] Lustre: 384245:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [13011.385386] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [13017.338115] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_default_debug -1 all [13018.706855] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [13018.727467] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [13018.877574] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [13026.967183] Lustre: DEBUG MARKER: oleg248-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [13034.468507] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [13034.472428] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [13034.477187] Lustre: Skipped 2 previous similar messages [13034.483259] Lustre: Skipped 8 previous similar messages [13036.882548] Lustre: server umount lustre-MDT0000 complete [13039.602400] LustreError: 382288:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [13039.648914] LustreError: 382288:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [13040.827486] LustreError: 382268:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788453596 with bad export cookie 13169643763217122979 [13040.842655] LustreError: 382268:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [13040.843522] LustreError: MGC192.168.202.148@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [13041.176959] Lustre: server umount lustre-MDT0001 complete [13056.838800] Lustre: server umount lustre-OST0000 complete [13086.807346] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing set_hostid [13100.092503] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing load_modules_local [13133.400240] Lustre: DEBUG MARKER: == conf-sanity test complete, duration 12815 sec ========= 12:41:27 (1788453687) [13135.279547] Lustre: DEBUG MARKER: === conf-sanity: start cleanup 12:41:29 (1788453689) === [13138.758112] Lustre: DEBUG MARKER: === conf-sanity: finish cleanup 12:41:33 (1788453693) === [13178.161656] Lustre: DEBUG MARKER: oleg248-server.virtnet: executing unload_modules_local [13181.367671] Key type lgssc unregistered [13181.821677] LNet: 390615:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [13181.831509] LNetError: 390615:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [13182.885426] LNet: Removed LNI 192.168.202.148@tcp [13183.999822] Key type .llcrypt unregistered [13184.006885] Key type ._llcrypt unregistered