[ 0.000000] Linux version 4.18.0rh8.10-debug (green@maintenance) (gcc version 8.5.0 20210514 (Red Hat 8.5.0-26) (GCC)) #2 SMP Mon Jul 14 01:24:22 EDT 2025 [ 0.000000] Command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers' [ 0.000000] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256 [ 0.000000] x86/fpu: Enabled xstate features 0x7, context size is 832 bytes, using 'standard' format. [ 0.000000] signal: max sigframe size: 1776 [ 0.000000] BIOS-provided physical RAM map: [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x00000000bffcdfff] usable [ 0.000000] BIOS-e820: [mem 0x00000000bffce000-0x00000000bfffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x0000000146dfffff] usable [ 0.000000] NX (Execute Disable) protection: active [ 0.000000] SMBIOS 3.0.0 present. [ 0.000000] DMI: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.16.3-2.fc40 04/01/2014 [ 0.000000] Hypervisor detected: KVM [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00 [ 0.000000] kvm-clock: using sched offset of 1219628624 cycles [ 0.000000] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns [ 0.000000] tsc: Detected 2399.998 MHz processor [ 0.000000] last_pfn = 0x146e00 max_arch_pfn = 0x400000000 [ 0.000000] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT [ 0.000000] last_pfn = 0xbffce max_arch_pfn = 0x400000000 [ 0.000000] found SMP MP-table at [mem 0x000f53f0-0x000f53ff] [ 0.000000] RAMDISK: [mem 0xbcc54000-0xbffbffff] [ 0.000000] ACPI: Early table checksum verification disabled [ 0.000000] ACPI: RSDP 0x00000000000F5200 000014 (v00 BOCHS ) [ 0.000000] ACPI: RSDT 0x00000000BFFE1D87 000034 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACP 0x00000000BFFE1C23 000074 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: DSDT 0x00000000BFFE0040 001BE3 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACS 0x00000000BFFE0000 000040 [ 0.000000] ACPI: APIC 0x00000000BFFE1C97 000090 (v03 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: HPET 0x00000000BFFE1D27 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: WAET 0x00000000BFFE1D5F 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: Reserving FACP table memory at [mem 0xbffe1c23-0xbffe1c96] [ 0.000000] ACPI: Reserving DSDT table memory at [mem 0xbffe0040-0xbffe1c22] [ 0.000000] ACPI: Reserving FACS table memory at [mem 0xbffe0000-0xbffe003f] [ 0.000000] ACPI: Reserving APIC table memory at [mem 0xbffe1c97-0xbffe1d26] [ 0.000000] ACPI: Reserving HPET table memory at [mem 0xbffe1d27-0xbffe1d5e] [ 0.000000] ACPI: Reserving WAET table memory at [mem 0xbffe1d5f-0xbffe1d86] [ 0.000000] No NUMA configuration found [ 0.000000] Faking a node at [mem 0x0000000000000000-0x0000000146dfffff] [ 0.000000] NODE_DATA(0) allocated [mem 0x1465a3000-0x1465cdfff] [ 0.000000] Reserving 256MB of memory at 2752MB for crashkernel (System RAM: 4205MB) [ 0.000000] Zone ranges: [ 0.000000] DMA [mem 0x0000000000001000-0x0000000000ffffff] [ 0.000000] DMA32 [mem 0x0000000001000000-0x00000000ffffffff] [ 0.000000] Normal [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Device empty [ 0.000000] Movable zone start for each node [ 0.000000] Early memory node ranges [ 0.000000] node 0: [mem 0x0000000000001000-0x000000000009efff] [ 0.000000] node 0: [mem 0x0000000000100000-0x00000000bffcdfff] [ 0.000000] node 0: [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Zeroed struct page in unavailable ranges: 4756 pages [ 0.000000] Initmem setup node 0 [mem 0x0000000000001000-0x0000000146dfffff] [ 0.000000] ACPI: PM-Timer IO Port: 0x608 [ 0.000000] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1]) [ 0.000000] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23 [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level) [ 0.000000] Using ACPI (MADT) for SMP configuration information [ 0.000000] ACPI: HPET id: 0x8086a201 base: 0xfed00000 [ 0.000000] TSC deadline timer available [ 0.000000] smpboot: Allowing 4 CPUs, 0 hotplug CPUs [ 0.000000] kvm-guest: KVM setup pv remote TLB flush [ 0.000000] kvm-guest: setup PV sched yield [ 0.000000] PM: Registered nosave memory: [mem 0x00000000-0x00000fff] [ 0.000000] PM: Registered nosave memory: [mem 0x0009f000-0x0009ffff] [ 0.000000] PM: Registered nosave memory: [mem 0x000a0000-0x000effff] [ 0.000000] PM: Registered nosave memory: [mem 0x000f0000-0x000fffff] [ 0.000000] PM: Registered nosave memory: [mem 0xbffce000-0xbfffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xc0000000-0xfeffbfff] [ 0.000000] PM: Registered nosave memory: [mem 0xfeffc000-0xfeffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xff000000-0xfffbffff] [ 0.000000] PM: Registered nosave memory: [mem 0xfffc0000-0xffffffff] [ 0.000000] [mem 0xc0000000-0xfeffbfff] available for PCI devices [ 0.000000] Booting paravirtualized kernel on KVM [ 0.000000] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns [ 0.000000] setup_percpu: NR_CPUS:8192 nr_cpumask_bits:4 nr_cpu_ids:4 nr_node_ids:1 [ 0.000000] percpu: Embedded 63 pages/cpu s221184 r8192 d28672 u524288 [ 0.000000] kvm-guest: PV spinlocks enabled [ 0.000000] PV qspinlock hash table entries: 256 (order: 0, 4096 bytes, linear) [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 1059606 [ 0.000000] Policy zone: Normal [ 0.000000] Kernel command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] Specific versions of hardware are certified with Red Hat Enterprise Linux 8. Please see the list of hardware certified with Red Hat Enterprise Linux 8 at https://catalog.redhat.com. [ 0.000000] audit: disabled (until reboot) [ 0.000000] software IO TLB: area num 4. [ 0.000000] Memory: 2829652K/4306352K available (18435K kernel code, 11221K rwdata, 7248K rodata, 2908K init, 18040K bss, 524584K reserved, 0K cma-reserved) [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1 [ 0.000000] kmemleak: Kernel memory leak detector disabled [ 0.000000] ftrace: allocating 41240 entries in 162 pages [ 0.000000] ftrace: allocated 162 pages with 3 groups [ 0.000000] rcu: Hierarchical RCU implementation. [ 0.000000] rcu: RCU event tracing is enabled. [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=8192 to nr_cpu_ids=4. [ 0.000000] rcu: RCU callback double-/use-after-free debug enabled. [ 0.000000] Rude variant of Tasks RCU enabled. [ 0.000000] Tracing variant of Tasks RCU enabled. [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies. [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4 [ 0.000000] NR_IRQS: 524544, nr_irqs: 456, preallocated irqs: 16 [ 0.000000] random: get_random_bytes called from start_kernel+0x622/0x9a8 with crng_init=0 [ 0.001000] Console: colour *CGA 80x25 [ 0.001000] printk: console [ttyS1] enabled [ 0.001000] ACPI: Core revision 20220331 [ 0.001000] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns [ 0.001014] APIC: Switch to symmetric I/O mode setup [ 0.002335] x2apic enabled [ 0.003015] Switched APIC routing to physical x2apic. [ 0.004024] kvm-guest: setup PV IPIs [ 0.011377] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1 [ 0.012020] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 0.014025] Calibrating delay loop (skipped) preset value.. 4799.99 BogoMIPS (lpj=2399998) [ 0.015020] pid_max: default: 32768 minimum: 301 [ 0.016174] LSM: Security Framework initializing [ 0.017092] Yama: becoming mindful. [ 0.018079] SELinux: Initializing. [ 0.019000] *** VALIDATE selinux *** [ 0.036104] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, vmalloc) [ 0.043220] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, vmalloc) [ 0.046066] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.047141] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.048147] *** VALIDATE tmpfs *** [ 0.050122] *** VALIDATE proc *** [ 0.053109] *** VALIDATE cgroup *** [ 0.054022] *** VALIDATE cgroup2 *** [ 0.055317] x86/cpu: User Mode Instruction Prevention (UMIP) activated [ 0.056191] Last level iTLB entries: 4KB 0, 2MB 0, 4MB 0 [ 0.057017] Last level dTLB entries: 4KB 0, 2MB 0, 4MB 0, 1GB 0 [ 0.058043] Spectre V2 : User space: Vulnerable [ 0.060022] Speculative Store Bypass: Vulnerable [ 0.064184] debug: unmapping init [mem 0xffffffff91a59000-0xffffffff91a60fff] [ 0.067286] smpboot: CPU0: Intel(R) Xeon(R) CPU E5-2695 v2 @ 2.40GHz (family: 0x6, model: 0x3e, stepping: 0x4) [ 0.068820] Performance Events: IvyBridge events, full-width counters, Intel PMU driver. [ 0.069032] ... version: 2 [ 0.070017] ... bit width: 48 [ 0.071027] ... generic registers: 4 [ 0.072016] ... value mask: 0000ffffffffffff [ 0.073023] ... max period: 00007fffffffffff [ 0.074026] ... fixed-purpose events: 3 [ 0.075022] ... event mask: 000000070000000f [ 0.077108] rcu: Hierarchical SRCU implementation. [ 0.080928] smp: Bringing up secondary CPUs ... [ 0.083138] x86: Booting SMP configuration: [ 0.084029] .... node #0, CPUs: #1 #2 #3 [ 0.093123] smp: Brought up 1 node, 4 CPUs [ 0.095043] smpboot: Max logical packages: 1 [ 0.096021] smpboot: Total of 4 processors activated (19199.98 BogoMIPS) [ 0.167039] node 0 deferred pages initialised in 67ms [ 0.179025] devtmpfs: initialized [ 0.182632] x86/mm: Memory block size: 128MB [ 0.188124] gcov: version magic: 0x41383552 [ 0.193785] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns [ 0.199118] futex hash table entries: 1024 (order: 4, 65536 bytes, vmalloc) [ 0.203406] pinctrl core: initialized pinctrl subsystem [ 0.205241] [ 0.206014] ************************************************************* [ 0.210021] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.213019] ** ** [ 0.214017] ** IOMMU DebugFS SUPPORT HAS BEEN ENABLED IN THIS KERNEL ** [ 0.218019] ** ** [ 0.222018] ** This means that this kernel is built to expose internal ** [ 0.226022] ** IOMMU data structures, which may compromise security on ** [ 0.230022] ** your system. ** [ 0.233019] ** ** [ 0.237024] ** If you see this message and you are not debugging the ** [ 0.242018] ** kernel, report this immediately to your vendor! ** [ 0.247019] ** ** [ 0.252018] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.256018] ************************************************************* [ 0.263332] NET: Registered protocol family 16 [ 0.268008] DMA: preallocated 512 KiB GFP_KERNEL pool for atomic allocations [ 0.272104] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations [ 0.277085] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations [ 0.285017] cpuidle: using governor menu [ 0.288465] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5 [ 0.295289] PCI: Using configuration type 1 for base access [ 0.298160] core: PMU erratum BJ122, BV98, HSD29 worked around, HT is on [ 0.310312] HugeTLB registered 1.00 GiB page size, pre-allocated 0 pages [ 0.311089] HugeTLB registered 2.00 MiB page size, pre-allocated 0 pages [ 0.322081] cryptd: max_cpu_qlen set to 1000 [ 0.333257] ACPI: Added _OSI(Module Device) [ 0.336020] ACPI: Added _OSI(Processor Device) [ 0.339022] ACPI: Added _OSI(3.0 _SCP Extensions) [ 0.341030] ACPI: Added _OSI(Processor Aggregator Device) [ 0.344363] ACPI: 1 ACPI AML tables successfully acquired and loaded [ 0.360864] ACPI: Interpreter enabled [ 0.363599] ACPI: PM: (supports S0 S3 S4 S5) [ 0.366018] ACPI: Using IOAPIC for interrupt routing [ 0.371188] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug [ 0.374444] ACPI: Enabled 2 GPEs in block 00 to 0F [ 0.397754] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff]) [ 0.401058] acpi PNP0A03:00: _OSC: OS supports [ASPM ClockPM Segments MSI HPX-Type3] [ 0.404025] acpi PNP0A03:00: _OSC: not requesting OS control; OS requires [ExtendedConfig ASPM ClockPM MSI] [ 0.407104] acpi PNP0A03:00: fail to add MMCONFIG information, can't access extended PCI configuration space under this bridge. [ 0.415269] acpiphp: Slot [2] registered [ 0.417243] acpiphp: Slot [5] registered [ 0.421142] acpiphp: Slot [6] registered [ 0.423237] acpiphp: Slot [7] registered [ 0.424000] acpiphp: Slot [8] registered [ 0.427116] acpiphp: Slot [9] registered [ 0.430200] acpiphp: Slot [10] registered [ 0.432243] acpiphp: Slot [3] registered [ 0.436166] acpiphp: Slot [4] registered [ 0.438167] acpiphp: Slot [11] registered [ 0.442157] acpiphp: Slot [12] registered [ 0.444158] acpiphp: Slot [13] registered [ 0.447173] acpiphp: Slot [14] registered [ 0.448176] acpiphp: Slot [15] registered [ 0.449207] acpiphp: Slot [16] registered [ 0.453182] acpiphp: Slot [17] registered [ 0.457130] acpiphp: Slot [18] registered [ 0.459160] acpiphp: Slot [19] registered [ 0.461146] acpiphp: Slot [20] registered [ 0.463157] acpiphp: Slot [21] registered [ 0.465251] acpiphp: Slot [22] registered [ 0.467113] acpiphp: Slot [23] registered [ 0.469095] acpiphp: Slot [24] registered [ 0.471101] acpiphp: Slot [25] registered [ 0.472104] acpiphp: Slot [26] registered [ 0.474131] acpiphp: Slot [27] registered [ 0.475250] acpiphp: Slot [28] registered [ 0.477134] acpiphp: Slot [29] registered [ 0.478574] acpiphp: Slot [30] registered [ 0.480097] acpiphp: Slot [31] registered [ 0.483081] PCI host bridge to bus 0000:00 [ 0.485034] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window] [ 0.489036] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window] [ 0.493034] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window] [ 0.499031] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window] [ 0.505031] pci_bus 0000:00: root bus resource [mem 0x380000000000-0x38007fffffff window] [ 0.508031] pci_bus 0000:00: root bus resource [bus 00-ff] [ 0.511082] pci 0000:00:00.0: [8086:1237] type 00 class 0x060000 [ 0.516102] pci 0000:00:01.0: [8086:7000] type 00 class 0x060100 [ 0.521025] pci 0000:00:01.1: [8086:7010] type 00 class 0x010180 [ 0.550954] pci 0000:00:01.1: reg 0x20: [io 0xc320-0xc32f] [ 0.566064] pci 0000:00:01.1: legacy IDE quirk: reg 0x10: [io 0x01f0-0x01f7] [ 0.574023] pci 0000:00:01.1: legacy IDE quirk: reg 0x14: [io 0x03f6] [ 0.584036] pci 0000:00:01.1: legacy IDE quirk: reg 0x18: [io 0x0170-0x0177] [ 0.593024] pci 0000:00:01.1: legacy IDE quirk: reg 0x1c: [io 0x0376] [ 0.604522] pci 0000:00:01.3: [8086:7113] type 00 class 0x068000 [ 0.613309] pci 0000:00:01.3: quirk: [io 0x0600-0x063f] claimed by PIIX4 ACPI [ 0.622046] pci 0000:00:01.3: quirk: [io 0x0700-0x070f] claimed by PIIX4 SMB [ 0.630101] pci 0000:00:01.3: quirk_piix4_acpi+0x0/0x1e0 took 15625 usecs [ 0.636778] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 [ 0.651021] pci 0000:00:02.0: reg 0x10: [io 0xc300-0xc31f] [ 0.684026] pci 0000:00:02.0: reg 0x20: [mem 0x380000000000-0x380000003fff 64bit pref] [ 0.697000] pci 0000:00:02.0: reg 0x30: [mem 0xfeb80000-0xfebbffff pref] [ 0.701575] pci 0000:00:05.0: [1af4:1001] type 00 class 0x010000 [ 0.722019] pci 0000:00:05.0: reg 0x10: [io 0xc000-0xc07f] [ 0.743019] pci 0000:00:05.0: reg 0x14: [mem 0xfebc0000-0xfebc0fff] [ 0.772023] pci 0000:00:05.0: reg 0x20: [mem 0x380000004000-0x380000007fff 64bit pref] [ 0.793255] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 [ 0.817033] pci 0000:00:06.0: reg 0x10: [io 0xc080-0xc0ff] [ 0.833023] pci 0000:00:06.0: reg 0x14: [mem 0xfebc1000-0xfebc1fff] [ 0.865020] pci 0000:00:06.0: reg 0x20: [mem 0x380000008000-0x38000000bfff 64bit pref] [ 0.896000] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 [ 0.909022] pci 0000:00:07.0: reg 0x10: [io 0xc100-0xc17f] [ 0.925022] pci 0000:00:07.0: reg 0x14: [mem 0xfebc2000-0xfebc2fff] [ 0.954021] pci 0000:00:07.0: reg 0x20: [mem 0x38000000c000-0x38000000ffff 64bit pref] [ 0.981328] pci 0000:00:08.0: [1af4:1001] type 00 class 0x010000 [ 0.991021] pci 0000:00:08.0: reg 0x10: [io 0xc180-0xc1ff] [ 1.005022] pci 0000:00:08.0: reg 0x14: [mem 0xfebc3000-0xfebc3fff] [ 1.043021] pci 0000:00:08.0: reg 0x20: [mem 0x380000010000-0x380000013fff 64bit pref] [ 1.067301] pci 0000:00:09.0: [1af4:1001] type 00 class 0x010000 [ 1.080037] pci 0000:00:09.0: reg 0x10: [io 0xc200-0xc27f] [ 1.096023] pci 0000:00:09.0: reg 0x14: [mem 0xfebc4000-0xfebc4fff] [ 1.130023] pci 0000:00:09.0: reg 0x20: [mem 0x380000014000-0x380000017fff 64bit pref] [ 1.146829] pci 0000:00:0a.0: [1af4:1001] type 00 class 0x010000 [ 1.165026] pci 0000:00:0a.0: reg 0x10: [io 0xc280-0xc2ff] [ 1.186024] pci 0000:00:0a.0: reg 0x14: [mem 0xfebc5000-0xfebc5fff] [ 1.216020] pci 0000:00:0a.0: reg 0x20: [mem 0x380000018000-0x38000001bfff 64bit pref] [ 1.236000] ACPI: PCI: Interrupt link LNKA configured for IRQ 10 [ 1.244494] ACPI: PCI: Interrupt link LNKB configured for IRQ 10 [ 1.250329] ACPI: PCI: Interrupt link LNKC configured for IRQ 11 [ 1.258280] ACPI: PCI: Interrupt link LNKD configured for IRQ 11 [ 1.264280] ACPI: PCI: Interrupt link LNKS configured for IRQ 9 [ 1.277054] iommu: Default domain type: Passthrough [ 1.278458] SCSI subsystem initialized [ 1.282063] ACPI: bus type USB registered [ 1.285185] usbcore: registered new interface driver usbfs [ 1.293119] usbcore: registered new interface driver hub [ 1.296118] usbcore: registered new device driver usb [ 1.298215] pps_core: LinuxPPS API ver. 1 registered [ 1.299000] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti [ 1.321103] PTP clock support registered [ 1.322120] EDAC MC: Ver: 3.0.0 [ 1.327150] PCI: Using ACPI for IRQ routing [ 1.333501] NetLabel: Initializing [ 1.336014] NetLabel: domain hash size = 128 [ 1.340014] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO [ 1.344103] NetLabel: unlabeled traffic allowed by default [ 1.349000] vgaarb: loaded [ 1.351426] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0 [ 1.352000] hpet0: 3 comparators, 64-bit 100.000000 MHz counter [ 1.361420] clocksource: Switched to clocksource kvm-clock [ 1.840844] VFS: Disk quotas dquot_6.6.0 [ 1.850178] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) [ 1.860397] *** VALIDATE ramfs *** [ 1.865218] *** VALIDATE hugetlbfs *** [ 1.871629] pnp: PnP ACPI init [ 1.882681] pnp: PnP ACPI: found 6 devices [ 1.953974] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns [ 2.104538] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window] [ 2.112492] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window] [ 2.120850] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window] [ 2.130210] pci_bus 0000:00: resource 7 [mem 0xc0000000-0xfebfffff window] [ 2.137082] pci_bus 0000:00: resource 8 [mem 0x380000000000-0x38007fffffff window] [ 2.147667] NET: Registered protocol family 2 [ 2.157767] IP idents hash table entries: 131072 (order: 8, 1048576 bytes, vmalloc) [ 2.180019] tcp_listen_portaddr_hash hash table entries: 4096 (order: 5, 163840 bytes, vmalloc) [ 2.195421] TCP established hash table entries: 65536 (order: 7, 524288 bytes, vmalloc) [ 2.220177] TCP bind hash table entries: 65536 (order: 9, 2097152 bytes, vmalloc) [ 2.237558] TCP: Hash tables configured (established 65536 bind 65536) [ 2.251646] MPTCP token hash table entries: 8192 (order: 6, 393216 bytes, vmalloc) [ 2.265274] UDP hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 2.278742] UDP-Lite hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 2.292487] NET: Registered protocol family 1 [ 2.308227] RPC: Registered named UNIX socket transport module. [ 2.316976] RPC: Registered udp transport module. [ 2.326373] RPC: Registered tcp transport module. [ 2.332900] RPC: Registered tcp NFSv4.1 backchannel transport module. [ 2.344127] NET: Registered protocol family 44 [ 2.354551] pci 0000:00:00.0: Limiting direct PCI/PCI transfers [ 2.361304] pci 0000:00:01.0: PIIX3: Enabling Passive Release [ 2.368215] pci 0000:00:01.0: Activating ISA DMA hang workarounds [ 2.377983] PCI: CLS 0 bytes, default 64 [ 2.383542] Unpacking initramfs... [ 3.721154] hrtimer: interrupt took 128906695 ns [ 10.652697] debug: unmapping init [mem 0xffff8b4c7cc54000-0xffff8b4c7ffbffff] [ 10.657321] PCI-DMA: Using software bounce buffering for IO (SWIOTLB) [ 10.659561] software IO TLB: mapped [mem 0x00000000a8000000-0x00000000ac000000] (64MB) [ 10.662530] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 14.422808] Initialise system trusted keyrings [ 14.424462] Key type blacklist registered [ 14.430579] workingset: timestamp_bits=36 max_order=20 bucket_order=0 [ 14.452010] zbud: loaded [ 14.468953] *** VALIDATE nfs *** [ 14.473687] *** VALIDATE nfs4 *** [ 14.477277] pstore: using deflate compression [ 14.488801] Platform Keyring initialized [ 15.072350] NET: Registered protocol family 38 [ 15.078721] Key type asymmetric registered [ 15.087513] Asymmetric key parser 'x509' registered [ 15.099935] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 247) [ 15.124937] io scheduler mq-deadline registered [ 15.126987] io scheduler kyber registered [ 15.132669] io scheduler bfq registered [ 15.138493] atomic64_test: passed for x86-64 platform with CX8 and with SSE [ 15.144217] shpchp: Standard Hot Plug PCI Controller Driver version: 0.4 [ 15.151840] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0 [ 15.161438] ACPI: Power Button [PWRF] [ 15.415988] ACPI: \_SB_.LNKB: Enabled at IRQ 10 [ 15.731399] ACPI: \_SB_.LNKA: Enabled at IRQ 11 [ 16.660339] ACPI: \_SB_.LNKC: Enabled at IRQ 11 [ 16.855439] ACPI: \_SB_.LNKD: Enabled at IRQ 10 [ 17.588807] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled [ 17.678761] 00:03: ttyS1 at I/O 0x2f8 (irq = 3, base_baud = 115200) is a 16550A [ 17.775142] 00:04: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A [ 17.795257] Non-volatile memory driver v1.3 [ 17.797721] Linux agpgart interface v0.103 [ 17.899374] virtio_blk virtio1: [vda] 133928 512-byte logical blocks (68.6 MB/65.4 MiB) [ 17.907557] vda: detected capacity change from 0 to 68571136 [ 18.199680] virtio_blk virtio2: [vdb] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) [ 18.211302] vdb: detected capacity change from 0 to 1073741824 [ 18.283813] virtio_blk virtio3: [vdc] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 18.320043] vdc: detected capacity change from 0 to 2621440000 [ 18.354713] virtio_blk virtio4: [vdd] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 18.366432] vdd: detected capacity change from 0 to 2621440000 [ 18.404413] virtio_blk virtio5: [vde] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 18.407680] vde: detected capacity change from 0 to 4294967296 [ 18.424513] virtio_blk virtio6: [vdf] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 18.427719] vdf: detected capacity change from 0 to 4294967296 [ 18.437970] libphy: Fixed MDIO Bus: probed [ 18.457742] usbcore: registered new interface driver usbserial_generic [ 18.461078] usbserial: USB Serial support registered for generic [ 18.463131] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12 [ 18.468138] serio: i8042 KBD port at 0x60,0x64 irq 1 [ 18.470142] serio: i8042 AUX port at 0x60,0x64 irq 12 [ 18.473533] mousedev: PS/2 mouse device common for all mice [ 18.476272] rtc_cmos 00:05: RTC can wake from S4 [ 18.479400] rtc_cmos 00:05: registered as rtc0 [ 18.481186] rtc_cmos 00:05: alarms up to one day, y3k, 242 bytes nvram, hpet irqs [ 18.483441] intel_pstate: CPU model not supported [ 18.504591] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input1 [ 18.510366] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input4 [ 18.514564] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input3 [ 18.511026] hpet1: lost 1 rtc interrupts [ 18.526558] hid: raw HID events driver (C) Jiri Kosina [ 18.529171] usbcore: registered new interface driver usbhid [ 18.531401] usbhid: USB HID core driver [ 18.533603] drop_monitor: Initializing network drop monitor service [ 18.543363] Initializing XFRM netlink socket [ 18.550591] NET: Registered protocol family 10 [ 18.566279] Segment Routing with IPv6 [ 18.567847] NET: Registered protocol family 17 [ 18.571743] mpls_gso: MPLS GSO support [ 18.577601] RAS: Correctable Errors collector initialized. [ 18.584397] AVX version of gcm_enc/dec engaged. [ 18.587544] AES CTR mode by8 optimization enabled [ 19.114232] sched_clock: Marking stable (19114174688, 0)->(23965008636, -4850833948) [ 19.117808] registered taskstats version 1 [ 19.120889] Loading compiled-in X.509 certificates [ 19.127215] zswap: loaded using pool lzo/zbud [ 19.225315] Key type big_key registered [ 19.289786] Key type encrypted registered [ 19.306091] ima: No TPM chip found, activating TPM-bypass! [ 19.322099] ima: Allocated hash algorithm: sha1 [ 19.334690] ima: No architecture policies found [ 19.345112] evm: Initialising EVM extended attributes: [ 19.350417] evm: security.selinux [ 19.353617] evm: security.ima [ 19.355308] evm: security.capability [ 19.364444] evm: HMAC attrs: 0x1 [ 19.374551] rtc_cmos 00:05: setting system clock to 2025-10-24 05:38:49 UTC (1761284329) [ 19.392536] debug: unmapping init [mem 0xffffffff92a03000-0xffffffff92bfffff] [ 19.400524] debug: unmapping init [mem 0xffffffff91782000-0xffffffff91a58fff] [ 19.404156] Write protecting the kernel read-only data: 28672k [ 19.411558] debug: unmapping init [mem 0xffffffff8fe03000-0xffffffff8fffffff] [ 19.418945] debug: unmapping init [mem 0xffffffff90714000-0xffffffff907fffff] [ 19.491926] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 19.512215] systemd[1]: Detected virtualization kvm. [ 19.520140] systemd[1]: Detected architecture x86-64. [ 19.529726] systemd[1]: Running in initial RAM disk. Welcome to Rocky Linux 8.10 (Green Obsidian) dracut-049-233.git20240115.el8 (Initramfs)! [ 19.581795] systemd[1]: No hostname configured. [ 19.586859] systemd[1]: Set hostname to . [ 19.591071] random: systemd: uninitialized urandom read (16 bytes read) [ 19.596724] systemd[1]: Initializing machine ID from random generator. [ 19.846344] random: ln: uninitialized urandom read (6 bytes read) [ 20.108136] random: systemd: uninitialized urandom read (16 bytes read) [ 20.113198] systemd[1]: Reached target Initrd Root Device. [ OK ] Reached target Initrd Root Device. [ 20.139209] systemd[1]: Reached target Timers. [ OK ] Reached target Timers. [ 20.219785] systemd[1]: Listening on udev Control Socket. [ OK ] Listening on udev Control Socket. [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Reached target Local Encrypted Volumes. [ OK ] Listening on Journal Socket (/dev/log). [ OK ] Reached target Slices. [ OK ] Reached target Swap. [ OK ] Reached target Local File Systems. [ OK ] Reached target Paths. [ OK ] Listening on Journal Socket. Starting Journal Service... Starting Setup Virtual Console... Starting Create Volatile Files and Directories... Starting Create list of required st…ce nodes for the current kernel... [ OK ] Started Memstrack Anylazing Service. [ OK ] Listening on udev Kernel Socket. Starting Apply Kernel Variables... [ OK ] Reached target Sockets. [ OK ] Started Setup Virtual Console. [ OK ] Started Create Volatile Files and Directories. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Started Apply Kernel Variables. Starting Create Static Device Nodes in /dev... Starting dracut cmdline hook... [ OK ] Started Create Static Device Nodes in /dev. [ OK ] Started Journal Service. [ OK ] Started dracut cmdline hook. Starting dracut pre-udev hook... [ 22.545616] device-mapper: uevent: version 1.0.3 [ 22.557345] device-mapper: ioctl: 4.46.0-ioctl (2022-02-22) initialised: dm-devel@redhat.com [ OK ] Started dracut pre-udev hook. Starting udev Kernel Device Manager... [ OK ] Started udev Kernel Device Manager. Starting dracut pre-trigger hook... [ OK ] Started dracut pre-trigger hook. Starting udev Coldplug all Devices... Mounting Kernel Configuration File System... [ OK ] Mounted Kernel Configuration File System. [ OK ] Started udev Coldplug all Devices. Starting dracut initqueue hook... [ OK ] Reached target System Initialization. [ OK ] Reached target Basic System. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. [ 24.457147] virtio_net virtio0 ens2: renamed from eth0 [ 24.762640] random: fast init done [ 25.062283] scsi host0: ata_piix [ 25.150748] scsi host1: ata_piix [ 25.156372] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc320 irq 14 [ 25.162088] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc328 irq 15 [ 29.923867] random: crng init done [ 29.925226] random: 7 urandom warning(s) missed due to ratelimiting [ 32.601150] dracut-initqueue[601]: RTNETLINK answers: File exists Starting nbd nbd0... [ OK ] Started nbd nbd0. [ OK ] Started dracut initqueue hook. Mounting /sysroot... [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. [ 34.987654] EXT4-fs (nbd0): mounted filesystem with ordered data mode. Opts: (null) [ OK ] Mounted /sysroot. [ OK ] Reached target Initrd Root File System. Starting Reload Configuration from the Real Root... [ OK ] Started Reload Configuration from the Real Root. [ OK ] Reached target Initrd File Systems. [ OK ] Reached target Initrd Default Target. Starting dracut pre-pivot and cleanup hook... [ OK ] Started dracut pre-pivot and cleanup hook. Starting Cleaning Up and Shutting Down Daemons... [ OK ] Stopped dracut pre-pivot and cleanup hook. [ OK ] Stopped target Timers. Stopping Hardware RNG Entropy Gatherer Daemon... [ OK ] Stopped target Remote File Systems. [ OK ] Stopped target Initrd Default Target. [ OK ] Stopped target Initrd Root Device. [ OK ] Stopped target Remote File Systems (Pre). [ OK ] Stopped dracut initqueue hook. [ OK ] Stopped Hardware RNG Entropy Gatherer Daemon. [ OK ] Stopped target Basic System. [ OK ] Stopped target System Initialization. [ OK ] Stopped udev Coldplug all Devices. [ OK ] Stopped dracut pre-trigger hook. [ OK ] Stopped target Swap. [ OK ] Stopped Create Volatile Files and Directories. [ OK ] Stopped Apply Kernel Variables. [ OK ] Stopped target Local File Systems. [ OK ] Stopped target Local Encrypted Volumes. Stopping udev Kernel Device Manager... [ OK ] Stopped target Paths. [ OK ] Stopped Dispatch Password Requests to Console Directory Watch. [ OK ] Stopped target Slices. [ OK ] Stopped target Sockets. [ OK ] Stopped udev Kernel Device Manager. [ OK ] Stopped Create Static Device Nodes in /dev. [ OK ] Stopped Create list of required sta…vice nodes for the current kernel. [ OK ] Stopped dracut pre-udev hook. [ OK ] Stopped dracut cmdline hook. [ OK ] Closed udev Control Socket. [ OK ] Closed udev Kernel Socket. Starting Cleanup udevd DB... [ OK ] Started Cleaning Up and Shutting Down Daemons. [ OK ] Started Cleanup udevd DB. [ OK ] Reached target Switch Root. Starting Switch Root... [ 38.784309] printk: systemd: 26 output lines suppressed due to ratelimiting [ 40.044208] SELinux: Disabled at runtime. [ 40.427347] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 40.438961] systemd[1]: Detected virtualization kvm. [ 40.446361] systemd[1]: Detected architecture x86-64. Welcome to Rocky Linux 8.10 (Green Obsidian)! [ 42.902966] systemd[1]: initrd-switch-root.service: Succeeded. [ 42.920444] systemd[1]: Stopped Switch Root. [ OK ] Stopped Switch Root. [ 42.947561] systemd[1]: systemd-journald.service: Service has no hold-off time (RestartSec=0), scheduling restart. [ 42.973987] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. [ 42.977987] systemd[1]: Stopped Journal Service. [ OK ] Stopped Journal Service. [ 43.021975] systemd[1]: Starting Journal Service... Starting Journal Service... [ 43.070395] systemd[1]: Starting Create list of required static device nodes for the current kernel... Starting Create list of required st…ce nodes for the current kernel... [ OK ] Created slice system-sshd\x2dkeygen.slice. Starting Remount Root and Kernel File Systems... [ OK ] Created slice system-getty.slice. Mounting Huge Pages File System... [ OK ] Listening on udev Control Socket. [ OK ] Listening on initctl Compatibility Named Pipe. [ OK ] Started Forward Password Requests to Wall Directory Watch. [ OK ] Reached target rpc_pipefs.target. [ OK ] Stopped target Switch Root. [ OK ] Stopped target Initrd File Systems. [ OK ] Stopped target Initrd Root File System. [ OK ] Created slice system-serial\x2dgetty.slice. [FAILED] Failed to set up automount Arbitrar…rmats File System Automount Point. See 'systemctl status proc-sys-fs-binfmt_misc.automount' for details. [ OK ] Listening on Process Core Dump Socket. Starting Apply Kernel Variables... [ OK ] Listening on udev Kernel Socket. Starting udev Coldplug all Devices... [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Reached target Paths. Activating swap /dev/disk/by-label/SWAP... Mounting POSIX Message Queue File System... Mounting Kernel Debug File System... [ OK ] Reached target Local Encrypted Volumes. [ OK ] Listening on RPCbind Server Activation Socket. [ OK ] Reached target RPC Port Mapper. [ OK ] Created slice User and Session Slice. [ OK ] Reached target Slices. [ 44.125406] Adding 1048572k swap on /dev/vdb. Priority:-2 extents:1 across:1048572k FS [ OK ] Started Journal Service. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [FAILED] Failed to start Remount Root and Kernel File Systems. See 'systemctl status systemd-remount-fs.service' for details. [ OK ] Mounted Huge Pages File System. [ OK ] Started Apply Kernel Variables. [ OK ] Activated swap /dev/disk/by-label/SWAP. [ OK ] Mounted POSIX Message Queue File System. [ OK ] Mounted Kernel Debug File System. [ OK ] Reached target Swap. Starting Configure read-only root support... Starting Create Static Device Nodes in /dev... Starting Flush Journal to Persistent Storage... [ OK ] Started udev Coldplug all Devices. [ OK ] Started Flush Journal to Persistent Storage. [ OK ] Started Create Static Device Nodes in /dev. [ OK ] Reached target Local File Systems (Pre). Mounting /home/green/git/lustre-release... Mounting /mnt... Starting udev Kernel Device Manager... [ OK ] Mounted /mnt. [ 45.231588] squashfs: version 4.0 (2009/01/31) Phillip Lougher [ OK ] Mounted /home/green/git/lustre-release. [ OK ] Started udev Kernel Device Manager. [ 47.298194] piix4_smbus 0000:00:01.3: SMBus Host Controller at 0x700, revision 0 [ 47.375152] input: PC Speaker as /devices/platform/pcspkr/input/input5 [ 50.682268] RAPL PMU: API unit is 2^-32 Joules, 0 fixed counters, 10737418240 ms ovfl timer [ 50.739142] EDAC sbridge: Ver: 1.1.2 [* ] A start job is running for Configur…-only root support (9s / no limit) [** ] A start job is running for Configur…-only root support (9s / no limit) [*** ] A start job is running for Configur…only root support (10s / no limit) [ *** ] A start job is running for Configur…only root support (10s / no limit) [ *** ] A start job is running for Configur…only root support (11s / no limit) [ ***] A start job is running for Configur…only root support (12s / no limit) [ **] A start job is running for Configur…only root support (12s / no limit) [ *] A start job is running for Configur…only root support (13s / no limit)[ 56.014630] Key type dns_resolver registered [ **] A start job is running for Configur…only root support (13s / no limit) [ ***] A start job is running for Configur…only root support (14s / no limit) [ *** ] A start job is running for Configur…only root support (14s / no limit)[ 57.396231] NFS: Registering the id_resolver key type [ 57.399307] Key type id_resolver registered [ 57.400806] Key type id_legacy registered [ *** ] A start job is running for Configur…only root support (15s / no limit) [ OK ] Started Configure read-only root support. [ OK ] Reached target Local File Systems. Starting Rebuild Dynamic Linker Cache... Starting Mark the need to relabel after reboot... Starting Create Volatile Files and Directories... Starting Load/Save Random Seed... [ OK ] Started Mark the need to relabel after reboot. [ OK ] Started Create Volatile Files and Directories. [ OK ] Started Load/Save Random Seed. Starting RPC Bind... Starting Update UTMP about System Boot/Shutdown... [ OK ] Started Update UTMP about System Boot/Shutdown. [ OK ] Started RPC Bind. [ OK ] Started Rebuild Dynamic Linker Cache. Starting Update is Completed... [ OK ] Started Update is Completed. [ OK ] Reached target System Initialization. [ OK ] Started Daily Cleanup of Temporary Directories. [ OK ] Started dnf makecache --timer. [ OK ] Listening on D-Bus System Message Bus Socket. [ OK ] Reached target Sockets. [ OK ] Reached target Basic System. [ OK ] Started irqbalance daemon. Starting Restore /run/initramfs on shutdown... [ OK ] Started Hardware RNG Entropy Gatherer Daemon. [ OK ] Started D-Bus System Message Bus. [ OK ] Reached target sshd-keygen.target. [ OK ] Started daily update of the root trust anchor for DNSSEC. [ OK ] Reached target Timers. Starting Network Manager... Starting Login Service... [ OK ] Started Restore /run/initramfs on shutdown. [ OK ] Started Network Manager. [ OK ] Reached target Network. Starting Dynamic System Tuning Daemon... Starting OpenSSH server daemon... Starting GSSAPI Proxy Daemon... Starting Network Manager Wait Online... Starting Hostname Service... [ OK ] Started OpenSSH server daemon. [ OK ] Started GSSAPI Proxy Daemon. [ OK ] Reached target NFS client services. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Starting Permit User Sessions... [ OK ] Started Permit User Sessions. [ OK ] Started Serial Getty on ttyS1. [ OK ] Started Command Scheduler. [ OK ] Started Serial Getty on ttyS0. [ OK ] Started Getty on tty1. [ OK ] Reached target Login Prompts. [ OK ] Started Login Service. [ OK ] Started Hostname Service. Starting Network Manager Script Dispatcher Service... [ OK ] Started Network Manager Script Dispatcher Service. [ OK ] Started Network Manager Wait Online. [ OK ] Reached target Network is Online. Starting System Logging Service... Starting Crash recovery kernel arming... Starting Notify NFS peers of a restart... [ OK ] Started Notify NFS peers of a restart. [ OK ] Started System Logging Service. Starting Authorization Manager... Rocky Linux 8.10 (Green Obsidian) Kernel 4.18.0rh8.10-debug on an x86_64 oleg152-server login: [ 96.432154] libcfs: loading out-of-tree module taints kernel. [ 96.444510] Key type ._llcrypt registered [ 96.445524] Key type .llcrypt registered [ 96.488716] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_hostid [ 106.380505] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing load_modules_local [ 106.989825] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 106.999646] alg: No test for adler32 (adler32-zlib) [ 108.018657] Lustre: Lustre: Build Version: 2.16.59_49_g4265256 [ 108.412226] LNet: Added LNI 192.168.201.152@tcp [8/256/0/180] [ 110.056169] Key type lgssc registered [ 110.996587] Lustre: Echo OBD driver; http://www.lustre.org/ [ 120.890235] ZFS: Loaded module v2.3.2-1, ZFS pool version 5000, ZFS filesystem version 5 [ 121.883883] LDISKFS-fs (vdc): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 127.118259] LDISKFS-fs (vdd): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 131.286868] LDISKFS-fs (vde): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 135.893914] LDISKFS-fs (vdf): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 145.013829] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing load_modules_local [ 151.732736] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 151.764732] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 151.780497] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 152.919623] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 152.952966] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 153.002173] Lustre: lustre-MDT0000: new disk, initializing [ 153.080928] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 153.092991] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 154.967171] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 161.935159] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 161.971323] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 162.015875] Lustre: 6517:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 162.035363] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 162.037645] Lustre: Skipped 1 previous similar message [ 162.082276] Lustre: lustre-MDT0001: new disk, initializing [ 162.117835] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 162.131599] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 162.138142] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 164.021228] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 166.791956] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 171.336197] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 171.379751] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 171.524141] Lustre: lustre-OST0000: new disk, initializing [ 171.529623] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 171.585613] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 174.448756] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 176.657790] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 176.671639] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 176.693939] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 181.336267] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 181.386860] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 181.441032] Lustre: lustre-OST0001: new disk, initializing [ 181.443706] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 181.472636] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 183.623354] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 183.630326] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 183.662038] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 184.949439] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 193.389300] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 197.847626] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 206.297929] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing check_logdir /tmp/testlogs/ [ 210.374099] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing yml_node [ 216.235780] Lustre: DEBUG MARKER: Client: 2.16.59.49 [ 219.247727] Lustre: DEBUG MARKER: MDS: 2.16.59.49 [ 222.165315] Lustre: DEBUG MARKER: OSS: 2.16.59.49 [ 223.981667] Lustre: DEBUG MARKER: -----============= acceptance-small: conf-sanity ============----- Fri Oct 24 01:42:12 EDT 2025 [ 239.886658] Lustre: DEBUG MARKER: excepting tests: 102 106 115 32newtarball [ 240.823597] Lustre: DEBUG MARKER: skipping tests SLOW=no: 45 69 106 111 114 [ 254.435889] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 254.443620] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 254.450842] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 255.457098] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 255.457257] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 255.464856] Lustre: Skipped 1 previous similar message [ 255.470570] Lustre: Skipped 3 previous similar messages [ 260.482211] LustreError: 12220:0:(obd_class.h:479:obd_check_dev()) Device 28 not setup [ 260.558306] Lustre: server umount lustre-MDT0000 complete [ 263.283922] LustreError: 7897:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1761284573 with bad export cookie 100877313984964579 [ 263.287587] LustreError: MGC192.168.201.152@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 263.296144] LustreError: 7897:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 265.697727] LustreError: 10043:0:(ldlm_lib.c:1245:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 265.698445] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 265.698631] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 265.715590] LustreError: 10043:0:(ldlm_lib.c:1245:target_handle_connect()) Skipped 6 previous similar messages [ 265.721444] Lustre: Skipped 2 previous similar messages [ 268.704973] LustreError: 6523:0:(ldlm_lib.c:1245:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 268.705860] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 268.719337] LustreError: 6523:0:(ldlm_lib.c:1245:target_handle_connect()) Skipped 1 previous similar message [ 268.720746] Lustre: Skipped 2 previous similar messages [ 269.476495] LustreError: 12422:0:(obd_class.h:479:obd_check_dev()) Device 27 not setup [ 269.483096] LustreError: 12422:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 269.643749] Lustre: server umount lustre-MDT0001 complete [ 272.281684] LustreError: 12623:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 272.290567] LustreError: 12623:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 272.331687] Lustre: server umount lustre-OST0000 complete [ 274.840573] LustreError: 12823:0:(obd_class.h:479:obd_check_dev()) Device 2 not setup [ 274.844225] LustreError: 12823:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 274.957722] Lustre: server umount lustre-OST0001 complete [ 279.405510] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_hostid [ 284.963140] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing load_modules_local [ 291.216867] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 296.052259] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 299.847678] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 304.417823] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 309.453569] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 309.512606] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 309.684237] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 309.732231] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 309.797986] Lustre: lustre-MDT0000: new disk, initializing [ 309.860777] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 309.876474] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 312.674915] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 320.107932] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 320.172602] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 320.240131] Lustre: 15004:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 320.296517] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 320.299699] Lustre: Skipped 1 previous similar message [ 320.350278] Lustre: lustre-MDT0001: new disk, initializing [ 320.424038] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 320.458463] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 320.470357] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 320.571598] Lustre: 15806:0:(osd_internal.h:1361:osd_trans_exec_op()) lustre-MDT0001: opcode 2: before 507 < left 524, rollback = 2 [ 320.581396] Lustre: 15806:0:(osd_handler.c:2088:osd_trans_dump_creds()) create: 3/12/6, destroy: 0/0/0 [ 320.590540] Lustre: 15806:0:(osd_handler.c:2095:osd_trans_dump_creds()) attr_set: 3/3/0, xattr_set: 5/524/0 [ 320.594625] Lustre: 15806:0:(osd_handler.c:2105:osd_trans_dump_creds()) write: 6/142/0, punch: 0/0/0, quota 1/3/0 [ 320.606515] Lustre: 15806:0:(osd_handler.c:2112:osd_trans_dump_creds()) insert: 7/135/3, delete: 0/0/0 [ 320.616612] Lustre: 15806:0:(osd_handler.c:2119:osd_trans_dump_creds()) ref_add: 1/1/0, ref_del: 0/0/0 [ 322.931840] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 328.255284] Lustre: DEBUG MARKER: oleg152-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 330.945159] Lustre: DEBUG MARKER: oleg152-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 335.193821] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 335.247549] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 336.627498] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 336.634359] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 336.751232] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 338.920149] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 344.281510] Lustre: DEBUG MARKER: oleg152-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 346.733183] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 346.883447] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 349.299837] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 349.426267] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 352.226385] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 352.227246] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 352.236395] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 352.242790] Lustre: Skipped 1 previous similar message [ 356.551203] LustreError: 17949:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 356.560181] LustreError: 17949:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 356.662702] Lustre: server umount lustre-OST0000 complete [ 361.444832] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 361.449568] LustreError: Skipped 1 previous similar message [ 361.452164] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 361.458223] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 361.460619] Lustre: Skipped 1 previous similar message [ 373.216122] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 373.299519] LustreError: 18151:0:(obd_class.h:479:obd_check_dev()) Device 22 not setup [ 373.302197] LustreError: 18151:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 373.353669] Lustre: server umount lustre-MDT0000 complete [ 375.566532] LustreError: 14998:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1761284685 with bad export cookie 100877313984966154 [ 375.569373] LustreError: MGC192.168.201.152@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 375.572060] LustreError: 14998:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 375.819153] Lustre: server umount lustre-MDT0001 complete [ 379.073761] Lustre: DEBUG MARKER: == conf-sanity test 76a: set permanent params with lctl across mounts ========================================================== 01:44:48 (1761284688) [ 383.161466] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 383.508199] LustreError: 18880:0:(ldlm_lib.c:1245:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 383.573517] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 383.577635] Lustre: Skipped 1 previous similar message [ 386.198216] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 388.578635] LustreError: 18880:0:(ldlm_lib.c:1245:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 391.455839] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 394.190521] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 397.790129] Lustre: DEBUG MARKER: oleg152-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 400.202992] Lustre: DEBUG MARKER: oleg152-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 404.701416] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 407.833206] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 410.714928] Lustre: DEBUG MARKER: oleg152-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 422.394989] Lustre: Modifying parameter general.osc.*.max_dirty_mb=452 in log params [ 427.506043] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 427.511537] Lustre: Skipped 1 previous similar message [ 427.515187] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 427.517874] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 427.532918] Lustre: Skipped 8 previous similar messages [ 427.663446] LustreError: 21204:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup [ 427.670627] LustreError: 21204:0:(obd_class.h:479:obd_check_dev()) Skipped 13 previous similar messages [ 427.735772] Lustre: server umount lustre-MDT0000 complete [ 432.157982] LustreError: 18860:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1761284742 with bad export cookie 100877313984967078 [ 432.169282] LustreError: MGC192.168.201.152@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 432.181584] LustreError: 18860:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 432.687651] Lustre: server umount lustre-MDT0001 complete [ 447.089379] Lustre: server umount lustre-OST0000 complete [ 470.578682] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing load_modules_local [ 479.433113] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 479.719143] LustreError: 22933:0:(ldlm_lib.c:1245:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 479.772733] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 479.776029] Lustre: Skipped 2 previous similar messages [ 482.621217] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 488.930537] LustreError: 22933:0:(ldlm_lib.c:1245:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 488.960186] LustreError: 22933:0:(ldlm_lib.c:1245:target_handle_connect()) Skipped 1 previous similar message [ 489.300664] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 492.510754] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 494.619495] Lustre: Setting parameter general.debug_raw_pointers=Y in log params [ 499.269745] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 504.172916] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 511.143109] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 511.215367] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 511.301829] Lustre: lustre-OST0001: new disk, initializing [ 511.304749] Lustre: Skipped 1 previous similar message [ 511.310911] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 511.315102] Lustre: Skipped 1 previous similar message [ 512.753600] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 512.764404] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 512.843873] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 515.783637] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 523.087133] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 525.974584] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 540.650076] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 540.651577] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 540.654292] Lustre: Skipped 2 previous similar messages [ 540.671198] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 540.681897] Lustre: Skipped 3 previous similar messages [ 540.699649] LustreError: 26417:0:(obd_class.h:479:obd_check_dev()) Device 28 not setup [ 540.704459] LustreError: 26417:0:(obd_class.h:479:obd_check_dev()) Skipped 17 previous similar messages [ 540.769809] Lustre: server umount lustre-MDT0000 complete [ 543.468375] LustreError: 23657:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1761284853 with bad export cookie 100877313984967918 [ 543.477258] LustreError: MGC192.168.201.152@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 543.481995] LustreError: 23657:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 543.754781] Lustre: server umount lustre-MDT0001 complete [ 557.244755] Lustre: server umount lustre-OST0000 complete [ 570.676449] Lustre: server umount lustre-OST0001 complete [ 581.744846] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing load_modules_local [ 589.610428] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 589.900484] LustreError: 28194:0:(ldlm_lib.c:1245:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 589.992068] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 590.002435] Lustre: Skipped 3 previous similar messages [ 593.069424] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 599.956590] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 603.753631] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 606.077960] Lustre: 29311:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 611.485326] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 616.713727] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 625.128737] LustreError: 29666:0:(ldlm_lib.c:1245:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 625.136212] LustreError: 29666:0:(ldlm_lib.c:1245:target_handle_connect()) Skipped 3 previous similar messages [ 625.341754] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 630.015442] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 635.876349] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 638.882625] Lustre: 31166:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 651.236473] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 651.244848] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 651.247676] Lustre: Skipped 4 previous similar messages [ 651.253835] Lustre: Skipped 3 previous similar messages [ 653.912670] LustreError: 31368:0:(obd_class.h:479:obd_check_dev()) Device 10 not setup [ 653.915166] LustreError: 31368:0:(obd_class.h:479:obd_check_dev()) Skipped 25 previous similar messages [ 653.978030] Lustre: server umount lustre-MDT0000 complete [ 656.821850] LustreError: 28173:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1761284966 with bad export cookie 100877313984969220 [ 656.833801] LustreError: MGC192.168.201.152@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 656.835905] LustreError: 28173:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 657.099722] Lustre: server umount lustre-MDT0001 complete [ 670.768708] Lustre: server umount lustre-OST0000 complete [ 683.133522] Lustre: server umount lustre-OST0001 complete [ 689.009772] Lustre: DEBUG MARKER: == conf-sanity test 76b: verify params log setup correctly ========================================================== 01:49:58 (1761284998) [ 708.597985] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing load_modules_local [ 715.004702] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 715.303690] LustreError: 33953:0:(ldlm_lib.c:1245:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 715.313953] LustreError: 33953:0:(ldlm_lib.c:1245:target_handle_connect()) Skipped 5 previous similar messages [ 717.649465] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 723.076850] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 723.343923] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 723.347580] Lustre: Skipped 4 previous similar messages [ 725.858799] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 727.930676] Lustre: 35073:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 732.897325] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 737.644277] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 743.780521] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 749.089259] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 756.181567] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 759.189737] Lustre: 36928:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 769.505435] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 769.512985] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 769.534226] Lustre: Skipped 2 previous similar messages [ 774.479585] Lustre: server umount lustre-MDT0000 complete [ 777.507105] LustreError: 33933:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1761285087 with bad export cookie 100877313984970326 [ 777.513627] LustreError: MGC192.168.201.152@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 777.858948] Lustre: server umount lustre-MDT0001 complete [ 790.038975] LustreError: 37484:0:(obd_class.h:479:obd_check_dev()) Device 23 not setup [ 790.043340] LustreError: 37484:0:(obd_class.h:479:obd_check_dev()) Skipped 41 previous similar messages [ 790.109353] Lustre: server umount lustre-OST0000 complete [ 803.860142] Lustre: server umount lustre-OST0001 complete [ 810.877723] Lustre: DEBUG MARKER: == conf-sanity test 76c: verify changelog_mask is applied with lctl set_param -P ========================================================== 01:52:00 (1761285120) [ 823.816227] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing load_modules_local [ 831.965561] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 835.096928] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 841.029253] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 844.020323] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 846.011601] Lustre: 40172:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 851.149577] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 852.517042] LustreError: 40529:0:(ldlm_lib.c:1245:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 852.524689] LustreError: 40529:0:(ldlm_lib.c:1245:target_handle_connect()) Skipped 11 previous similar messages [ 856.082919] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 862.137896] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 866.267442] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 871.206439] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 878.859464] Lustre: Setting parameter general.mdd.*.changelog_mask=-CLOSE in log params [ 878.867310] Lustre: Skipped 1 previous similar message [ 882.145898] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 882.150851] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 882.153289] Lustre: Skipped 4 previous similar messages [ 889.161440] Lustre: server umount lustre-MDT0000 complete [ 891.830322] LustreError: 40174:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1761285201 with bad export cookie 100877313984971432 [ 891.831854] LustreError: MGC192.168.201.152@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 891.835392] LustreError: 40174:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 906.208136] Lustre: lustre-MDT0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 906.487277] Lustre: server umount lustre-MDT0001 complete [ 912.651800] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 915.939787] LustreError: 3659:0:(client.c:1380:ptlrpc_import_delay_req()) @@@ invalidate in flight req@ffff8b4cff81dc00 x1846840570258176/t0(0) o250->MGC192.168.201.152@tcp@0@lo:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl Rpc:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 920.203439] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 927.349928] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 931.023297] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 931.769356] Lustre: lustre-MDT0000-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 935.427974] Lustre: DEBUG MARKER: oleg152-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 936.529376] Lustre: DEBUG MARKER: mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec [ 939.129816] Lustre: DEBUG MARKER: oleg152-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 940.339626] Lustre: DEBUG MARKER: mdc.lustre-MDT0001-mdc-*.mds_server_uuid in FULL state after 0 sec [ 947.174650] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 947.182788] Lustre: Skipped 10 previous similar messages [ 949.588677] Lustre: server umount lustre-MDT0000 complete [ 952.268807] LustreError: 39035:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1761285262 with bad export cookie 100877313984972622 [ 952.271770] LustreError: MGC192.168.201.152@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 952.283867] LustreError: 39035:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 952.577286] Lustre: server umount lustre-MDT0001 complete [ 961.200517] Lustre: server umount lustre-OST0000 complete [ 964.116409] Lustre: server umount lustre-OST0001 complete [ 970.207038] Lustre: DEBUG MARKER: == conf-sanity test 76d: verify llite.*.xattr_cache can be set by 'lctl set_param -P' correctly ========================================================== 01:54:39 (1761285279) [ 981.456476] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing load_modules_local [ 989.394482] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 989.754461] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 989.757727] Lustre: Skipped 8 previous similar messages [ 992.474188] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 998.517362] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1001.228489] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 1003.413491] Lustre: 47523:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1003.418178] Lustre: 47523:0:(mgs_llog.c:1346:mgs_modify_param()) Skipped 1 previous similar message [ 1009.051922] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1014.577159] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 1022.045138] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1027.142938] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 1033.666508] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1043.220529] Lustre: Setting parameter general.llite.*.xattr_cache=0 in log params [ 1057.317735] LustreError: 49600:0:(obd_class.h:479:obd_check_dev()) Device 10 not setup [ 1057.320503] LustreError: 49600:0:(obd_class.h:479:obd_check_dev()) Skipped 51 previous similar messages [ 1057.391805] Lustre: server umount lustre-MDT0000 complete [ 1061.542483] LustreError: 47140:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1761285371 with bad export cookie 100877313984973399 [ 1061.547147] LustreError: MGC192.168.201.152@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1061.552873] LustreError: 47140:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1061.970304] Lustre: server umount lustre-MDT0001 complete [ 1075.005180] Lustre: server umount lustre-OST0000 complete [ 1088.413626] Lustre: server umount lustre-OST0001 complete [ 1095.949411] Lustre: DEBUG MARKER: == conf-sanity test 77: comma-separated MGS NIDs and failover node NIDs ========================================================== 01:56:45 (1761285405) [ 1098.178135] Lustre: DEBUG MARKER: SKIP: conf-sanity test_77 mixed loopback and real device not working [ 1099.737392] Lustre: DEBUG MARKER: == conf-sanity test 78: run resize2fs on MDT and OST filesystems ========================================================== 01:56:48 (1761285408) [ 1109.256141] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_hostid [ 1116.051936] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing load_modules_local [ 1123.755536] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1129.162297] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1135.149733] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1135.216340] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1135.427108] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1135.480399] Lustre: lustre-MDT0000: new disk, initializing [ 1135.574123] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1139.577948] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 1145.412261] Lustre: DEBUG MARKER: oleg152-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1150.993542] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1151.040076] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1152.947147] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 1152.990551] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 1156.203768] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 1162.702594] Lustre: DEBUG MARKER: oleg152-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1164.491438] Lustre: DEBUG MARKER: create test files [ 1188.835796] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1188.836167] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1188.843201] Lustre: Skipped 24 previous similar messages [ 1191.203708] Lustre: server umount lustre-OST0000 complete [ 1193.769330] Lustre: server umount lustre-MDT0000 complete [ 1200.043651] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing unload_modules_local [ 1201.889448] Key type lgssc unregistered [ 1202.090893] LNet: 54941:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1202.096190] LNetError: 54941:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1203.113502] LNet: Removed LNI 192.168.201.152@tcp [ 1203.691663] Key type .llcrypt unregistered [ 1203.695562] Key type ._llcrypt unregistered [ 1220.010879] Key type ._llcrypt registered [ 1220.012463] Key type .llcrypt registered [ 1220.116788] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing load_modules_local [ 1220.925228] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1220.935290] alg: No test for adler32 (adler32-zlib) [ 1221.911853] Lustre: Lustre: Build Version: 2.16.59_49_g4265256 [ 1222.073319] LNet: Added LNI 192.168.201.152@tcp [8/256/0/180] [ 1223.720498] Key type lgssc registered [ 1224.444681] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1229.727210] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1229.761935] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1231.167223] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1234.083276] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 1237.899145] Lustre: DEBUG MARKER: oleg152-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1242.931391] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1243.204022] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1246.693831] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 1250.044535] Lustre: DEBUG MARKER: oleg152-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1256.495686] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:100 to 0x240000400:129) [ 1256.576154] Lustre: 57524:0:(osd_internal.h:1361:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 515 < left 544, rollback = 9 [ 1256.583690] Lustre: 57524:0:(osd_handler.c:2088:osd_trans_dump_creds()) create: 32/128/0, destroy: 0/0/0 [ 1256.587966] Lustre: 57524:0:(osd_handler.c:2095:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 1256.592665] Lustre: 57524:0:(osd_handler.c:2105:osd_trans_dump_creds()) write: 2/13/1, punch: 0/0/0, quota 1/3/0 [ 1256.596487] Lustre: 57524:0:(osd_handler.c:2112:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 1256.600851] Lustre: 57524:0:(osd_handler.c:2119:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 1264.336303] Lustre: DEBUG MARKER: check files after expanding the MDT and OST filesystems [ 1267.434636] Lustre: DEBUG MARKER: create more files after expanding the MDT and OST filesystems [ 1271.779884] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1271.780523] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1271.791520] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1276.899363] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1276.992149] LustreError: 58291:0:(obd_class.h:479:obd_check_dev()) Device 13 not setup [ 1277.155775] Lustre: server umount lustre-OST0000 complete [ 1279.616835] LustreError: 58493:0:(obd_class.h:479:obd_check_dev()) Device 8 not setup [ 1279.623163] LustreError: 58493:0:(obd_class.h:479:obd_check_dev()) Skipped 1 previous similar message [ 1279.763901] Lustre: server umount lustre-MDT0000 complete [ 1287.481556] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing unload_modules_local [ 1289.222713] Key type lgssc unregistered [ 1289.434351] LNet: 59023:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1289.443643] LNetError: 59023:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1289.457993] LNet: Removed LNI 192.168.201.152@tcp [ 1289.930122] Key type .llcrypt unregistered [ 1289.931482] Key type ._llcrypt unregistered [ 1308.870282] Key type ._llcrypt registered [ 1308.871683] Key type .llcrypt registered [ 1308.939421] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing load_modules_local [ 1309.398461] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1309.449557] alg: No test for adler32 (adler32-zlib) [ 1310.441122] Lustre: Lustre: Build Version: 2.16.59_49_g4265256 [ 1310.670594] LNet: Added LNI 192.168.201.152@tcp [8/256/0/180] [ 1312.288128] Key type lgssc registered [ 1313.312682] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1319.755806] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1319.767130] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1321.121859] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1323.763586] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 1327.961202] Lustre: DEBUG MARKER: oleg152-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1332.585084] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1332.770826] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1336.528477] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 1340.670182] Lustre: DEBUG MARKER: oleg152-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1346.034075] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:140 to 0x240000400:161) [ 1346.079702] Lustre: 61703:0:(osd_internal.h:1361:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 515 < left 544, rollback = 9 [ 1346.086785] Lustre: 61703:0:(osd_handler.c:2088:osd_trans_dump_creds()) create: 32/128/0, destroy: 0/0/0 [ 1346.093032] Lustre: 61703:0:(osd_handler.c:2095:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 1346.098973] Lustre: 61703:0:(osd_handler.c:2105:osd_trans_dump_creds()) write: 2/13/1, punch: 0/0/0, quota 1/3/0 [ 1346.102293] Lustre: 61703:0:(osd_handler.c:2112:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 1346.105797] Lustre: 61703:0:(osd_handler.c:2119:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 1350.070440] Lustre: DEBUG MARKER: check files after shrinking the MDT and OST filesystems [ 1386.979364] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1386.981769] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1387.003524] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1392.103201] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1393.042453] LustreError: 62438:0:(obd_class.h:479:obd_check_dev()) Device 13 not setup [ 1393.227509] Lustre: server umount lustre-OST0000 complete [ 1396.308670] LustreError: 62640:0:(obd_class.h:479:obd_check_dev()) Device 8 not setup [ 1396.323666] LustreError: 62640:0:(obd_class.h:479:obd_check_dev()) Skipped 1 previous similar message [ 1396.524663] Lustre: server umount lustre-MDT0000 complete [ 1405.445557] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing unload_modules_local [ 1407.619630] Key type lgssc unregistered [ 1407.841916] LNet: 63171:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1407.849855] LNetError: 63171:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1407.860702] LNet: Removed LNI 192.168.201.152@tcp [ 1408.408461] Key type .llcrypt unregistered [ 1408.410435] Key type ._llcrypt unregistered [ 1422.975512] Key type ._llcrypt registered [ 1422.977172] Key type .llcrypt registered [ 1423.069754] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_hostid [ 1435.335726] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing load_modules_local [ 1436.143908] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1436.207965] alg: No test for adler32 (adler32-zlib) [ 1437.239107] Lustre: Lustre: Build Version: 2.16.59_49_g4265256 [ 1437.458992] LNet: Added LNI 192.168.201.152@tcp [8/256/0/180] [ 1439.104202] Key type lgssc registered [ 1440.076852] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1447.166100] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1452.823834] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1457.733598] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1464.675691] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1471.501152] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1471.557707] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1471.568926] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1472.803374] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 1472.853187] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1472.914075] Lustre: lustre-MDT0000: new disk, initializing [ 1472.979297] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1472.994966] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1476.058970] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 1484.839152] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1484.905304] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1484.977661] Lustre: 66278:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 1485.022475] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 1485.026312] Lustre: Skipped 1 previous similar message [ 1485.074327] Lustre: lustre-MDT0001: new disk, initializing [ 1485.161958] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 1485.199552] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 1485.204803] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 1485.273631] Lustre: 67074:0:(osd_internal.h:1361:osd_trans_exec_op()) lustre-MDT0001: opcode 2: before 507 < left 524, rollback = 2 [ 1485.284321] Lustre: 67074:0:(osd_handler.c:2088:osd_trans_dump_creds()) create: 3/12/6, destroy: 0/0/0 [ 1485.291367] Lustre: 67074:0:(osd_handler.c:2095:osd_trans_dump_creds()) attr_set: 3/3/0, xattr_set: 5/524/0 [ 1485.300311] Lustre: 67074:0:(osd_handler.c:2105:osd_trans_dump_creds()) write: 6/142/0, punch: 0/0/0, quota 1/3/0 [ 1485.309726] Lustre: 67074:0:(osd_handler.c:2112:osd_trans_dump_creds()) insert: 7/135/3, delete: 0/0/0 [ 1485.316100] Lustre: 67074:0:(osd_handler.c:2119:osd_trans_dump_creds()) ref_add: 1/1/0, ref_del: 0/0/0 [ 1488.640451] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 1495.413124] Lustre: DEBUG MARKER: oleg152-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1499.049623] Lustre: DEBUG MARKER: oleg152-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 1504.672610] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1504.711928] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1504.979338] Lustre: lustre-OST0000: new disk, initializing [ 1504.983661] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1505.074382] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1509.607257] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 1510.490295] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 1510.495289] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 1510.603865] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 1515.184841] Lustre: DEBUG MARKER: oleg152-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1517.749631] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 1517.859749] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 1520.697320] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 1520.850154] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 1524.207895] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1524.212431] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1524.227719] Lustre: Skipped 1 previous similar message [ 1524.238720] Lustre: Skipped 1 previous similar message [ 1528.509074] LustreError: 69218:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 1528.629644] Lustre: server umount lustre-OST0000 complete [ 1534.434859] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1534.440454] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1534.444126] Lustre: Skipped 1 previous similar message [ 1539.555896] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1539.566845] Lustre: Skipped 2 previous similar messages [ 1544.675713] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1544.699281] Lustre: Skipped 1 previous similar message [ 1546.208204] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 1546.292545] LustreError: 69421:0:(obd_class.h:479:obd_check_dev()) Device 22 not setup [ 1546.296432] LustreError: 69421:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 1546.428451] Lustre: server umount lustre-MDT0000 complete [ 1550.093741] LustreError: 67083:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1761285860 with bad export cookie 14654698406389599127 [ 1550.097807] LustreError: MGC192.168.201.152@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1550.110909] LustreError: 67083:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1550.340695] LustreError: 69624:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 1550.344853] LustreError: 69624:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 1550.663744] Lustre: server umount lustre-MDT0001 complete [ 1558.610912] Lustre: DEBUG MARKER: == conf-sanity test 79: format MDT/OST without mgs option (should return errors) ========================================================== 02:04:27 (1761285867) [ 1567.540533] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing load_modules_local [ 1575.147208] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1594.144848] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_hostid [ 1601.245130] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing load_modules_local [ 1608.244345] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1613.823953] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1619.052272] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1624.435164] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1630.882428] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1630.948725] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1631.177880] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 1631.226701] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1631.399750] Lustre: lustre-MDT0000: new disk, initializing [ 1631.507817] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1631.525324] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1635.924420] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 1646.473663] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1646.560382] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1646.702141] Lustre: 73862:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 1646.800916] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 1646.811174] Lustre: Skipped 1 previous similar message [ 1646.905841] Lustre: lustre-MDT0001: Not available for connect from 0@lo (not set up) [ 1646.930908] Lustre: lustre-MDT0001: new disk, initializing [ 1647.010606] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 1647.085227] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 1647.106335] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 1647.202624] Lustre: 74664:0:(osd_internal.h:1361:osd_trans_exec_op()) lustre-MDT0001: opcode 2: before 507 < left 524, rollback = 2 [ 1647.217373] Lustre: 74664:0:(osd_internal.h:1361:osd_trans_exec_op()) Skipped 1 previous similar message [ 1647.235654] Lustre: 74664:0:(osd_handler.c:2088:osd_trans_dump_creds()) create: 3/12/6, destroy: 0/0/0 [ 1647.252732] Lustre: 74664:0:(osd_handler.c:2088:osd_trans_dump_creds()) Skipped 1 previous similar message [ 1647.259749] Lustre: 74664:0:(osd_handler.c:2095:osd_trans_dump_creds()) attr_set: 3/3/0, xattr_set: 5/524/0 [ 1647.275433] Lustre: 74664:0:(osd_handler.c:2095:osd_trans_dump_creds()) Skipped 1 previous similar message [ 1647.290643] Lustre: 74664:0:(osd_handler.c:2105:osd_trans_dump_creds()) write: 6/142/0, punch: 0/0/0, quota 1/3/0 [ 1647.302346] Lustre: 74664:0:(osd_handler.c:2105:osd_trans_dump_creds()) Skipped 1 previous similar message [ 1647.325882] Lustre: 74664:0:(osd_handler.c:2112:osd_trans_dump_creds()) insert: 7/135/3, delete: 0/0/0 [ 1647.336460] Lustre: 74664:0:(osd_handler.c:2112:osd_trans_dump_creds()) Skipped 1 previous similar message [ 1647.345866] Lustre: 74664:0:(osd_handler.c:2119:osd_trans_dump_creds()) ref_add: 1/1/0, ref_del: 0/0/0 [ 1647.356235] Lustre: 74664:0:(osd_handler.c:2119:osd_trans_dump_creds()) Skipped 1 previous similar message [ 1650.244347] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 1656.795450] Lustre: DEBUG MARKER: oleg152-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1659.520792] Lustre: DEBUG MARKER: oleg152-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 1664.737512] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1664.799356] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1665.001545] Lustre: lustre-OST0000: new disk, initializing [ 1665.004172] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1665.073442] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1668.353289] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 1673.725771] Lustre: DEBUG MARKER: oleg152-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1673.781909] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 1673.793719] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 1673.881033] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 1676.621559] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 1676.768484] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 1679.416498] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 1679.599574] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 1684.450496] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 1684.452051] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1684.462675] LustreError: Skipped 1 previous similar message [ 1684.482288] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1684.485073] Lustre: Skipped 1 previous similar message [ 1695.200995] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 1695.237034] LustreError: 76804:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 1695.244208] LustreError: 76804:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 1695.383203] Lustre: server umount lustre-OST0000 complete [ 1698.273338] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1698.286073] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1698.293960] Lustre: Skipped 1 previous similar message [ 1704.370372] LustreError: 77007:0:(obd_class.h:479:obd_check_dev()) Device 22 not setup [ 1704.376175] LustreError: 77007:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 1704.485498] Lustre: server umount lustre-MDT0000 complete [ 1704.931241] LustreError: 74998:0:(ldlm_lib.c:1245:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1704.953680] LustreError: 74998:0:(ldlm_lib.c:1245:target_handle_connect()) Skipped 4 previous similar messages [ 1707.849216] LustreError: 73852:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1761286017 with bad export cookie 14654698406389600051 [ 1707.855723] LustreError: MGC192.168.201.152@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1707.867235] LustreError: 73852:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 1708.229809] Lustre: server umount lustre-MDT0001 complete [ 1714.432459] Lustre: DEBUG MARKER: == conf-sanity test 80: mgc import reconnect race ======== 02:07:03 (1761286023) [ 1719.643348] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1720.031582] LustreError: 77879:0:(ldlm_lib.c:1245:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1720.090109] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1723.000033] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 1725.414482] LustreError: 77879:0:(ldlm_lib.c:1245:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1729.237289] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1732.726399] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 1737.569521] Lustre: DEBUG MARKER: oleg152-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1741.162027] Lustre: DEBUG MARKER: oleg152-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 1748.544673] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1755.111720] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 1760.088563] Lustre: DEBUG MARKER: oleg152-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1762.535375] Lustre: 80221:0:(genops.c:1791:obd_export_evict_by_uuid()) MGS: evicting f63bbe72-0c9f-40a1-b8c6-854e7632dfad at adminstrative request [ 1764.327606] LustreError: MGC192.168.201.152@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1764.344952] Lustre: *** cfs_fail_loc=906, val=2147483648*** [ 1774.516940] Lustre: MGS: Client f63bbe72-0c9f-40a1-b8c6-854e7632dfad (at 0@lo) reconnecting [ 1774.525492] Lustre: Evicted from MGS (at 0@lo) after server handle changed from 0xcb5ff28e8355d2cf to 0xcb5ff28e8355d568 [ 1774.551168] Lustre: MGC192.168.201.152@tcp: Connection restored to 0@lo (at 0@lo) [ 1799.038351] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1799.118694] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1799.260602] Lustre: lustre-OST0001: new disk, initializing [ 1799.265515] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 1799.375689] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 1799.386204] Lustre: Skipped 2 previous similar messages [ 1803.265795] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 1805.379724] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 1805.391866] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 1805.453854] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 1809.357555] Lustre: DEBUG MARKER: oleg152-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid [ 1816.035257] LustreError: lustre-OST0001-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 1816.038564] Lustre: lustre-OST0001-osc-MDT0000: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1816.042776] LustreError: Skipped 1 previous similar message [ 1816.048660] Lustre: Skipped 1 previous similar message [ 1816.051851] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 1816.051858] Lustre: Skipped 7 previous similar messages [ 1817.662100] LustreError: 81357:0:(obd_class.h:479:obd_check_dev()) Device 25 not setup [ 1817.671116] LustreError: 81357:0:(obd_class.h:479:obd_check_dev()) Skipped 13 previous similar messages [ 1817.729784] Lustre: server umount lustre-OST0001 complete [ 1821.157048] LustreError: 79482:0:(ldlm_lib.c:1245:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1821.174316] LustreError: 79482:0:(ldlm_lib.c:1245:target_handle_connect()) Skipped 1 previous similar message [ 1826.277165] LustreError: 80233:0:(ldlm_lib.c:1245:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1826.288481] LustreError: 80233:0:(ldlm_lib.c:1245:target_handle_connect()) Skipped 1 previous similar message [ 1826.878654] Lustre: server umount lustre-OST0000 complete [ 1831.400058] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1831.412758] Lustre: Skipped 4 previous similar messages [ 1835.546410] LustreError: 81761:0:(obd_class.h:479:obd_check_dev()) Device 28 not setup [ 1835.554135] LustreError: 81761:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 1835.642836] Lustre: server umount lustre-MDT0000 complete [ 1836.514230] LustreError: 77878:0:(ldlm_lib.c:1245:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1836.522810] LustreError: 77878:0:(ldlm_lib.c:1245:target_handle_connect()) Skipped 1 previous similar message [ 1838.192491] LustreError: 77860:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1761286148 with bad export cookie 14654698406389601640 [ 1838.198497] LustreError: 77860:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 1 previous similar message [ 1838.200682] LustreError: MGC192.168.201.152@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1838.408881] Lustre: server umount lustre-MDT0001 complete [ 1844.177229] Lustre: DEBUG MARKER: == conf-sanity test 81: sparse OST indexing ============== 02:09:13 (1761286153) [ 1845.409059] Lustre: DEBUG MARKER: SKIP: conf-sanity test_81 needs >= 3 OSTs [ 1846.761396] Lustre: DEBUG MARKER: == conf-sanity test 82a: specify OSTs for file (succeed) or directory (succeed) ========================================================== 02:09:15 (1761286155) [ 1848.079589] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82a needs >= 3 OSTs [ 1849.387225] Lustre: DEBUG MARKER: == conf-sanity test 82b: specify OSTs for file with --pool and --ost-list options ========================================================== 02:09:18 (1761286158) [ 1850.522847] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82b needs >= 4 OSTs [ 1851.746349] Lustre: DEBUG MARKER: == conf-sanity test 83: ENOSPACE on OST doesn't cause message VFS: Busy inodes after unmount ... ========================================================== 02:09:21 (1761286161) [ 1852.997045] Lustre: DEBUG MARKER: mount the OST /dev/mapper/ost1_flakey as a ldiskfs filesystem [ 1856.350602] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1857.057218] loop: module loaded [ 1857.071233] loop0: detected capacity change from 0 to 4294967296 [ 1857.103130] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 1858.247430] Lustre: DEBUG MARKER: run llverfs in partial mode on the OST ldiskfs /mnt/lustre-ost1 [ 1860.349491] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing run_llverfs /mnt/lustre-ost1 -vpl no [ 1863.044319] Lustre: DEBUG MARKER: unmount the OST /dev/mapper/ost1_flakey [ 1869.119602] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1872.008815] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1875.253702] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1878.282733] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1883.158841] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1883.210814] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1883.232798] LustreError: 84534:0:(obd_config.c:783:class_setup()) setup lustre-OST0000-osd failed (-28) [ 1883.237522] LustreError: 84534:0:(obd_mount.c:193:lustre_start_simple()) lustre-OST0000-osd setup error -28 [ 1883.241163] LustreError: 84534:0:(tgt_mount.c:2450:server_fill_super()) Unable to start osd on /dev/mapper/ost1_flakey: -28 [ 1883.247322] LustreError: 84534:0:(super25.c:199:lustre_fill_super()) llite: Unable to mount : rc = -28 [ 1896.760213] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_hostid [ 1903.009572] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing load_modules_local [ 1910.345381] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1914.519776] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1918.615145] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1923.257159] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1928.077526] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1928.133515] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1928.306144] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 1928.330033] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1928.377686] Lustre: lustre-MDT0000: new disk, initializing [ 1928.439638] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1928.455299] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1930.600495] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 1938.182308] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1938.235349] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1938.294996] Lustre: 87375:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 1938.362649] Lustre: lustre-MDT0001: Not available for connect from 0@lo (not set up) [ 1938.365435] Lustre: Skipped 7 previous similar messages [ 1938.504786] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 1938.577638] Lustre: 88177:0:(osd_internal.h:1361:osd_trans_exec_op()) lustre-MDT0001: opcode 2: before 507 < left 524, rollback = 2 [ 1938.588634] Lustre: 88177:0:(osd_internal.h:1361:osd_trans_exec_op()) Skipped 1 previous similar message [ 1938.596259] Lustre: 88177:0:(osd_handler.c:2088:osd_trans_dump_creds()) create: 3/12/6, destroy: 0/0/0 [ 1938.606436] Lustre: 88177:0:(osd_handler.c:2088:osd_trans_dump_creds()) Skipped 1 previous similar message [ 1938.613516] Lustre: 88177:0:(osd_handler.c:2095:osd_trans_dump_creds()) attr_set: 3/3/0, xattr_set: 5/524/0 [ 1938.618524] Lustre: 88177:0:(osd_handler.c:2095:osd_trans_dump_creds()) Skipped 1 previous similar message [ 1938.626156] Lustre: 88177:0:(osd_handler.c:2105:osd_trans_dump_creds()) write: 6/142/0, punch: 0/0/0, quota 1/3/0 [ 1938.644369] Lustre: 88177:0:(osd_handler.c:2105:osd_trans_dump_creds()) Skipped 1 previous similar message [ 1938.653871] Lustre: 88177:0:(osd_handler.c:2112:osd_trans_dump_creds()) insert: 7/135/3, delete: 0/0/0 [ 1938.665574] Lustre: 88177:0:(osd_handler.c:2112:osd_trans_dump_creds()) Skipped 1 previous similar message [ 1938.669558] Lustre: 88177:0:(osd_handler.c:2119:osd_trans_dump_creds()) ref_add: 1/1/0, ref_del: 0/0/0 [ 1938.684555] Lustre: 88177:0:(osd_handler.c:2119:osd_trans_dump_creds()) Skipped 1 previous similar message [ 1941.733395] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 1947.557428] Lustre: DEBUG MARKER: oleg152-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1950.336893] Lustre: DEBUG MARKER: oleg152-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 1955.498341] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1955.568654] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1956.873415] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 1957.033171] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 1961.395139] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 1967.496218] Lustre: DEBUG MARKER: oleg152-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1971.160347] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 1971.405236] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 1974.889312] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 1975.053938] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 1977.314325] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1977.319705] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1977.343446] Lustre: Skipped 1 previous similar message [ 1991.136144] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 1991.175149] LustreError: 90316:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 1991.182931] LustreError: 90316:0:(obd_class.h:479:obd_check_dev()) Skipped 17 previous similar messages [ 1991.323876] Lustre: server umount lustre-OST0000 complete [ 2001.185085] Lustre: server umount lustre-MDT0000 complete [ 2002.914488] LustreError: 88179:0:(ldlm_lib.c:1245:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2002.924123] LustreError: 88179:0:(ldlm_lib.c:1245:target_handle_connect()) Skipped 1 previous similar message [ 2004.239382] LustreError: 87367:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1761286314 with bad export cookie 14654698406389602242 [ 2004.245775] LustreError: MGC192.168.201.152@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2004.271653] LustreError: 87367:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 5 previous similar messages [ 2004.546466] Lustre: server umount lustre-MDT0001 complete [ 2012.827914] Lustre: DEBUG MARKER: == conf-sanity test 84: check recovery_hard_time ========= 02:12:01 (1761286321) [ 2019.839386] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2024.296475] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 2032.533866] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2036.607440] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 2042.143892] Lustre: DEBUG MARKER: oleg152-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2045.598703] Lustre: DEBUG MARKER: oleg152-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2051.920331] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2057.556242] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 2063.041694] Lustre: DEBUG MARKER: oleg152-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2067.329109] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2067.605815] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2071.030450] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2071.203681] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2076.659912] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2076.743477] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2076.874540] Lustre: lustre-OST0001: new disk, initializing [ 2076.876366] Lustre: Skipped 2 previous similar messages [ 2076.889630] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 2076.892526] Lustre: Skipped 3 previous similar messages [ 2077.009192] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 2077.015857] Lustre: Skipped 5 previous similar messages [ 2083.095181] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 2083.497497] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 2083.511843] Lustre: Skipped 2 previous similar messages [ 2083.523282] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 2083.665080] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 2090.950045] Lustre: DEBUG MARKER: oleg152-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid [ 2095.360161] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid 50 [ 2095.629782] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2100.021866] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [ 2100.205081] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2119.570197] Lustre: DEBUG MARKER: mds1 REPLAY BARRIER on lustre-MDT0000 [ 2147.679702] Lustre: Failing over lustre-MDT0000 [ 2148.006905] LustreError: 96304:0:(obd_class.h:479:obd_check_dev()) Device 28 not setup [ 2148.012305] LustreError: 96304:0:(obd_class.h:479:obd_check_dev()) Skipped 17 previous similar messages [ 2148.090623] Lustre: server umount lustre-MDT0000 complete [ 2148.975838] LustreError: 91388:0:(ldlm_lib.c:1245:target_handle_connect()) lustre-MDT0000: not available for connect from 192.168.201.52@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2149.001063] LustreError: 91388:0:(ldlm_lib.c:1245:target_handle_connect()) Skipped 4 previous similar messages [ 2150.375539] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2150.388283] Lustre: Skipped 4 previous similar messages [ 2166.754768] Lustre: 64489:0:(client.c:2468:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1761286460/real 1761286460] req@00000000a39fab18 x1846841963710848/t0(0) o400->MGC192.168.201.152@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1761286476 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 2166.775724] LustreError: MGC192.168.201.152@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2170.896329] LDISKFS-fs (dm-0): 4 truncates cleaned up [ 2170.898897] LDISKFS-fs (dm-0): recovery complete [ 2170.912406] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2175.970568] LustreError: 64488:0:(client.c:1380:ptlrpc_import_delay_req()) @@@ invalidate in flight req@0000000012907d4e x1846841963718784/t0(0) o250->MGC192.168.201.152@tcp@0@lo:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl Rpc:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 2176.191519] Lustre: lustre-MDT0000: in recovery but waiting for the first client to connect [ 2179.448670] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 2180.699834] Lustre: lustre-MDT0000: Will be in recovery for at least 1:00, or until 3 clients reconnect [ 2180.702765] Lustre: 92905:0:(ldlm_lib.c:2133:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 2181.606902] Lustre: 91393:0:(ldlm_lib.c:2133:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 2181.634993] Lustre: 91393:0:(ldlm_lib.c:2133:extend_recovery_timer()) Skipped 1 previous similar message [ 2181.650616] Lustre: lustre-MDT0000-lwp-OST0001: Connection restored to 0@lo (at 0@lo) [ 2181.746582] LustreError: 97053:0:(ldlm_lib.c:2745:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2182.064168] LustreError: 97053:0:(ldlm_lib.c:2745:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2182.406757] LustreError: 97053:0:(ldlm_lib.c:2745:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2182.410373] LustreError: 97053:0:(ldlm_lib.c:2745:replay_request_or_update()) Skipped 1 previous similar message [ 2182.738291] LustreError: 97053:0:(ldlm_lib.c:2745:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2182.748199] LustreError: 97053:0:(ldlm_lib.c:2745:replay_request_or_update()) Skipped 1 previous similar message [ 2182.763123] Lustre: 97053:0:(ldlm_lib.c:2133:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2182.776167] Lustre: 97053:0:(ldlm_lib.c:2133:extend_recovery_timer()) Skipped 7 previous similar messages [ 2183.425243] LustreError: 97053:0:(ldlm_lib.c:2745:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2183.437500] LustreError: 97053:0:(ldlm_lib.c:2745:replay_request_or_update()) Skipped 2 previous similar messages [ 2183.760388] LustreError: 97053:0:(ldlm_lib.c:2745:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2183.766064] LustreError: 97053:0:(ldlm_lib.c:2745:replay_request_or_update()) Skipped 2 previous similar messages [ 2185.064365] Lustre: 97053:0:(ldlm_lib.c:2133:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2185.071923] Lustre: 97053:0:(ldlm_lib.c:2133:extend_recovery_timer()) Skipped 6 previous similar messages [ 2185.721815] LustreError: 97053:0:(ldlm_lib.c:2745:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2185.740243] LustreError: 97053:0:(ldlm_lib.c:2745:replay_request_or_update()) Skipped 6 previous similar messages [ 2186.065306] LustreError: 97053:0:(ldlm_lib.c:2745:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2186.076877] LustreError: 97053:0:(ldlm_lib.c:2745:replay_request_or_update()) Skipped 6 previous similar messages [ 2189.260726] Lustre: 97053:0:(ldlm_lib.c:2133:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2189.280187] Lustre: 97053:0:(ldlm_lib.c:2133:extend_recovery_timer()) Skipped 12 previous similar messages [ 2189.923649] LustreError: 97053:0:(ldlm_lib.c:2745:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2189.935289] LustreError: 97053:0:(ldlm_lib.c:2745:replay_request_or_update()) Skipped 12 previous similar messages [ 2190.256404] LustreError: 97053:0:(ldlm_lib.c:2745:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2190.269068] LustreError: 97053:0:(ldlm_lib.c:2745:replay_request_or_update()) Skipped 12 previous similar messages [ 2197.269641] Lustre: 97053:0:(ldlm_lib.c:2133:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2197.276146] Lustre: 97053:0:(ldlm_lib.c:2133:extend_recovery_timer()) Skipped 24 previous similar messages [ 2198.233739] LustreError: 97053:0:(ldlm_lib.c:2745:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2198.240713] LustreError: 97053:0:(ldlm_lib.c:2745:replay_request_or_update()) Skipped 25 previous similar messages [ 2198.560869] LustreError: 97053:0:(ldlm_lib.c:2745:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2198.571244] LustreError: 97053:0:(ldlm_lib.c:2745:replay_request_or_update()) Skipped 25 previous similar messages [ 2213.448966] Lustre: 97053:0:(ldlm_lib.c:2133:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2213.470424] Lustre: 97053:0:(ldlm_lib.c:2133:extend_recovery_timer()) Skipped 50 previous similar messages [ 2214.421586] LustreError: 97053:0:(ldlm_lib.c:2745:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2214.431407] LustreError: 97053:0:(ldlm_lib.c:2745:replay_request_or_update()) Skipped 50 previous similar messages [ 2214.752287] LustreError: 97053:0:(ldlm_lib.c:2745:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2214.757869] LustreError: 97053:0:(ldlm_lib.c:2745:replay_request_or_update()) Skipped 50 previous similar messages [ 2240.217834] Lustre: 97053:0:(ldlm_lib.c:2441:target_recovery_overseer()) lustre-MDT0000 recovery is aborted by hard timeout [ 2240.225972] Lustre: 97053:0:(ldlm_lib.c:2451:target_recovery_overseer()) recovery is aborted, evict exports in recovery [ 2240.235724] Lustre: 97053:0:(genops.c:1618:class_disconnect_stale_exports()) lustre-MDT0000: disconnect stale client 8b86efea-5bf7-4187-8f48-26513675082a@192.168.201.52@tcp [ 2240.252459] Lustre: lustre-MDT0000: disconnecting 1 stale clients [ 2240.258138] Lustre: 97053:0:(ldlm_lib.c:1963:abort_req_replay_queue()) @@@ aborted: req@00000000c11af992 x1846841950935680/t0(8589934784) o35->8b86efea-5bf7-4187-8f48-26513675082a@192.168.201.52@tcp:666/0 lens 392/0 e 0 to 0 dl 1761286561 ref 1 fl Complete:/604/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:0 [ 2240.276313] Lustre: lustre-MDT0000-osd: cancel update llog [0x200000400:0x1:0x0] [ 2240.294858] Lustre: lustre-MDT0000: Denying connection for new client 8b86efea-5bf7-4187-8f48-26513675082a (at 192.168.201.52@tcp), waiting for 3 known clients (2 recovered, 0 in progress, and 1 evicted) already passed deadline 0:00 [ 2240.307445] Lustre: lustre-MDT0001-osp-MDT0000: cancel update llog [0x240000401:0x1:0x0] [ 2240.365951] Lustre: lustre-MDT0000-osp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 2240.378782] Lustre: Skipped 2 previous similar messages [ 2240.382740] Lustre: lustre-MDT0000: Recovery over after 1:00, of 3 clients 2 recovered and 1 was evicted. [ 2240.406765] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:80 to 0x280000401:577) [ 2240.410524] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:48 to 0x2c0000401:545) [ 2252.258864] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2252.269841] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2252.277896] Lustre: Skipped 1 previous similar message [ 2252.281634] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2252.284489] Lustre: Skipped 8 previous similar messages [ 2257.768074] Lustre: server umount lustre-OST0000 complete [ 2258.404541] LustreError: 92994:0:(ldlm_lib.c:1245:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2258.436205] LustreError: 92994:0:(ldlm_lib.c:1245:target_handle_connect()) Skipped 41 previous similar messages [ 2261.473787] LustreError: lustre-OST0001-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2267.469823] Lustre: server umount lustre-OST0001 complete [ 2276.389268] LustreError: 98118:0:(obd_class.h:479:obd_check_dev()) Device 10 not setup [ 2276.391744] LustreError: 98118:0:(obd_class.h:479:obd_check_dev()) Skipped 15 previous similar messages [ 2276.465929] Lustre: server umount lustre-MDT0000 complete [ 2279.049278] LustreError: 91374:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1761286589 with bad export cookie 14654698406389653503 [ 2279.054715] LustreError: 91374:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 1 previous similar message [ 2279.055103] LustreError: MGC192.168.201.152@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2279.351561] Lustre: server umount lustre-MDT0001 complete [ 2286.003452] Lustre: DEBUG MARKER: == conf-sanity test 85: osd_ost init: fail ea_fid_set ==== 02:16:35 (1761286595) [ 2292.337859] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2292.348193] Lustre: *** cfs_fail_loc=197, val=0*** [ 2307.872613] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 2312.992496] LustreError: 99012:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.201.152@tcp: failed processing log, type 4: rc = -110 [ 2318.152699] Lustre: *** cfs_fail_loc=197, val=0*** [ 2318.155101] Lustre: Skipped 65 previous similar messages [ 2338.720535] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2338.733148] Lustre: Skipped 1 previous similar message [ 2343.827827] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 2348.240706] Lustre: DEBUG MARKER: oleg152-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2349.940523] Lustre: server umount lustre-OST0000 complete [ 2356.627939] Lustre: DEBUG MARKER: == conf-sanity test 86: Replacing mkfs.lustre -G option == 02:17:45 (1761286665) [ 2374.861973] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2378.209744] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 2384.399222] Lustre: DEBUG MARKER: oleg152-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2388.093691] LustreError: 100551:0:(osp_object.c:617:osp_attr_get()) lustre-MDT0001-osp-MDT0000: osp_attr_get update error [0x200000009:0x1:0x0]: rc = -5 [ 2388.100756] LustreError: 100551:0:(lod_sub_object.c:917:lod_sub_prep_llog()) lustre-MDT0000-mdtlov: can't get id from catalogs: rc = -5 [ 2388.104891] LustreError: 100551:0:(lod_dev.c:510:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 13, retries 0, failed: rc = -5 [ 2388.472093] Lustre: server umount lustre-MDT0000 complete [ 2401.937699] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2417.641951] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_hostid [ 2424.816974] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing load_modules_local [ 2433.199560] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2439.054061] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2444.167350] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2448.860847] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2453.994805] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2454.042228] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2454.222196] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2454.270803] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2454.324919] Lustre: lustre-MDT0000: new disk, initializing [ 2454.487501] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2457.765219] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 2465.888847] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2465.931304] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2465.990217] Lustre: 104592:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2466.190318] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2466.311359] Lustre: 105394:0:(osd_internal.h:1361:osd_trans_exec_op()) lustre-MDT0001: opcode 2: before 507 < left 524, rollback = 2 [ 2466.323798] Lustre: 105394:0:(osd_internal.h:1361:osd_trans_exec_op()) Skipped 1 previous similar message [ 2466.331024] Lustre: 105394:0:(osd_handler.c:2088:osd_trans_dump_creds()) create: 3/12/6, destroy: 0/0/0 [ 2466.338329] Lustre: 105394:0:(osd_handler.c:2088:osd_trans_dump_creds()) Skipped 1 previous similar message [ 2466.345654] Lustre: 105394:0:(osd_handler.c:2095:osd_trans_dump_creds()) attr_set: 3/3/0, xattr_set: 5/524/0 [ 2466.350800] Lustre: 105394:0:(osd_handler.c:2095:osd_trans_dump_creds()) Skipped 1 previous similar message [ 2466.355221] Lustre: 105394:0:(osd_handler.c:2105:osd_trans_dump_creds()) write: 6/142/0, punch: 0/0/0, quota 1/3/0 [ 2466.366978] Lustre: 105394:0:(osd_handler.c:2105:osd_trans_dump_creds()) Skipped 1 previous similar message [ 2466.374574] Lustre: 105394:0:(osd_handler.c:2112:osd_trans_dump_creds()) insert: 7/135/3, delete: 0/0/0 [ 2466.386278] Lustre: 105394:0:(osd_handler.c:2112:osd_trans_dump_creds()) Skipped 1 previous similar message [ 2466.390406] Lustre: 105394:0:(osd_handler.c:2119:osd_trans_dump_creds()) ref_add: 1/1/0, ref_del: 0/0/0 [ 2466.400535] Lustre: 105394:0:(osd_handler.c:2119:osd_trans_dump_creds()) Skipped 1 previous similar message [ 2468.800046] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 2473.426393] Lustre: DEBUG MARKER: oleg152-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2475.916665] Lustre: DEBUG MARKER: oleg152-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2480.649353] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2480.723206] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2485.235388] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 2489.030943] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2491.595122] Lustre: DEBUG MARKER: oleg152-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2494.499513] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2494.661785] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2497.638834] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2497.833564] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2499.555281] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2499.556887] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2499.566031] LustreError: Skipped 2 previous similar messages [ 2499.581741] Lustre: Skipped 5 previous similar messages [ 2499.586441] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2499.599462] Lustre: Skipped 9 previous similar messages [ 2505.728381] Lustre: server umount lustre-OST0000 complete [ 2514.760696] Lustre: server umount lustre-MDT0000 complete [ 2514.917298] LustreError: 104599:0:(ldlm_lib.c:1245:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2514.926420] LustreError: 104599:0:(ldlm_lib.c:1245:target_handle_connect()) Skipped 8 previous similar messages [ 2517.571216] LustreError: 104585:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1761286827 with bad export cookie 14654698406389664171 [ 2517.572348] LustreError: MGC192.168.201.152@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2517.577685] LustreError: 104585:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 2517.838856] Lustre: server umount lustre-MDT0001 complete [ 2524.619418] Lustre: DEBUG MARKER: == conf-sanity test 87: check if MDT inode can hold EAs with N stripes properly ========================================================== 02:20:33 (1761286833) [ 2531.406433] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing unload_modules_local [ 2533.697952] Key type lgssc unregistered [ 2534.023732] LNet: 108665:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2534.028908] LNetError: 108665:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2534.048491] LNet: Removed LNI 192.168.201.152@tcp [ 2534.721321] Key type .llcrypt unregistered [ 2534.726386] Key type ._llcrypt unregistered [ 2550.402640] Key type ._llcrypt registered [ 2550.404561] Key type .llcrypt registered [ 2550.566927] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_hostid [ 2566.641707] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing load_modules_local [ 2568.099776] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2568.128743] alg: No test for adler32 (adler32-zlib) [ 2569.314388] Lustre: Lustre: Build Version: 2.16.59_49_g4265256 [ 2569.640231] LNet: Added LNI 192.168.201.152@tcp [8/256/0/180] [ 2571.312264] Key type lgssc registered [ 2572.577934] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2582.690315] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2590.543309] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2597.550889] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2605.107888] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2610.397769] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2616.723348] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2616.785275] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 2616.800724] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2618.038332] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2618.070877] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2618.123905] Lustre: lustre-MDT0000: new disk, initializing [ 2618.215771] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2618.238070] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2621.918845] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 2631.789291] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2631.871312] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2632.205748] Lustre: lustre-OST0000: new disk, initializing [ 2632.235314] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 2632.240298] Lustre: Skipped 1 previous similar message [ 2632.329060] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2633.115761] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 2633.123374] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 2633.171513] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 2638.032557] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 2648.531325] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2648.583366] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2648.690135] Lustre: lustre-OST0001: new disk, initializing [ 2648.693396] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 2648.792139] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 2654.754727] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 2657.914708] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:1:ost [ 2657.925138] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:1:ost] [ 2657.967402] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x280000400 [ 2668.007755] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2668.021493] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2668.033103] Lustre: Skipped 1 previous similar message [ 2669.205566] LustreError: 114247:0:(obd_class.h:479:obd_check_dev()) Device 17 not setup [ 2669.418286] Lustre: server umount lustre-MDT0000 complete [ 2676.243765] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: (null) [ 2680.248364] LustreError: 114248:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1761286990 with bad export cookie 18396878256719072714 [ 2680.250266] LustreError: MGC192.168.201.152@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2680.261024] LustreError: 114248:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 1 previous similar message [ 2680.298112] LustreError: 114845:0:(obd_class.h:479:obd_check_dev()) Device 12 not setup [ 2680.300371] LustreError: 114845:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 2680.346308] Lustre: server umount lustre-OST0000 complete [ 2683.360928] LustreError: 115046:0:(obd_class.h:479:obd_check_dev()) Device 2 not setup [ 2683.372117] LustreError: 115046:0:(obd_class.h:479:obd_check_dev()) Skipped 1 previous similar message [ 2683.515550] Lustre: server umount lustre-OST0001 complete [ 2690.645929] Lustre: DEBUG MARKER: == conf-sanity test 88: check the default mount options can be overridden ========================================================== 02:23:19 (1761286999) [ 2697.788869] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing load_modules_local [ 2705.326294] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2706.303966] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2710.745601] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2711.579270] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2716.272760] Lustre: DEBUG MARKER: == conf-sanity test 89: check tunefs --param and --erase-param{s} options ========================================================== 02:23:45 (1761287025) [ 2725.559427] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2727.065608] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2728.585263] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2730.161838] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2731.059195] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2732.978490] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2748.361890] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_hostid [ 2757.321951] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing load_modules_local [ 2766.723159] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2773.377608] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2779.518067] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2785.740594] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2791.812420] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2791.869536] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2792.069428] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2792.098543] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2792.148729] Lustre: lustre-MDT0000: new disk, initializing [ 2792.214979] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2792.227281] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2795.866723] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 2806.869103] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2806.969728] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2807.163500] Lustre: 120369:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2807.286957] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 2807.292070] Lustre: Skipped 1 previous similar message [ 2807.349394] Lustre: lustre-MDT0001: new disk, initializing [ 2807.449839] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2807.487149] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 2807.498453] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2807.557773] Lustre: 121169:0:(osd_internal.h:1361:osd_trans_exec_op()) lustre-MDT0001: opcode 2: before 507 < left 524, rollback = 2 [ 2807.568602] Lustre: 121169:0:(osd_handler.c:2088:osd_trans_dump_creds()) create: 3/12/6, destroy: 0/0/0 [ 2807.577951] Lustre: 121169:0:(osd_handler.c:2095:osd_trans_dump_creds()) attr_set: 3/3/0, xattr_set: 5/524/0 [ 2807.589656] Lustre: 121169:0:(osd_handler.c:2105:osd_trans_dump_creds()) write: 6/142/0, punch: 0/0/0, quota 1/3/0 [ 2807.603429] Lustre: 121169:0:(osd_handler.c:2112:osd_trans_dump_creds()) insert: 7/135/3, delete: 0/0/0 [ 2807.623131] Lustre: 121169:0:(osd_handler.c:2119:osd_trans_dump_creds()) ref_add: 1/1/0, ref_del: 0/0/0 [ 2812.002403] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 2820.484989] Lustre: DEBUG MARKER: oleg152-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2824.533478] Lustre: DEBUG MARKER: oleg152-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2832.463768] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2832.528777] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2832.862832] Lustre: lustre-OST0000: new disk, initializing [ 2832.866272] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 2832.986668] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2838.732829] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 2839.630899] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 2839.644877] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2839.807722] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2845.689171] Lustre: DEBUG MARKER: oleg152-client.virtnet: executing wait_import_state_mount (FULL|IDLE) failover.node=192.0.2.254@tcp0,osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2849.738229] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2849.951579] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2853.927822] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2854.214488] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2860.515461] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2860.515648] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2860.529906] LustreError: Skipped 1 previous similar message [ 2860.537788] Lustre: Skipped 2 previous similar messages [ 2860.540338] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2865.639326] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2865.653431] Lustre: Skipped 2 previous similar messages [ 2870.243446] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2870.316109] LustreError: 123313:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 2870.321189] LustreError: 123313:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 2870.489702] Lustre: server umount lustre-OST0000 complete [ 2874.343308] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2874.355984] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2874.388755] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2880.363217] LustreError: 123518:0:(obd_class.h:479:obd_check_dev()) Device 22 not setup [ 2880.368022] LustreError: 123518:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 2880.466319] Lustre: server umount lustre-MDT0000 complete [ 2883.895417] LustreError: 120364:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1761287194 with bad export cookie 18396878256719073960 [ 2883.899750] LustreError: MGC192.168.201.152@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2883.902160] LustreError: 120364:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 2884.282722] Lustre: server umount lustre-MDT0001 complete [ 2890.640742] Lustre: DEBUG MARKER: == conf-sanity test 90a: check max_mod_rpcs_in_flight is enforced ========================================================== 02:26:39 (1761287199) [ 2895.437683] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2895.821599] LustreError: 124391:0:(ldlm_lib.c:1245:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2895.828634] LustreError: 124391:0:(ldlm_lib.c:1245:target_handle_connect()) Skipped 3 previous similar messages [ 2895.890951] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2899.174956] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 2900.978329] LustreError: 124390:0:(ldlm_lib.c:1245:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2905.062173] LustreError: 124390:0:(ldlm_lib.c:1245:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2906.343635] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2910.353762] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 2915.744073] Lustre: DEBUG MARKER: oleg152-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2918.857370] Lustre: DEBUG MARKER: oleg152-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2924.894924] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2929.869596] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 2934.515556] Lustre: DEBUG MARKER: oleg152-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2948.856471] Lustre: *** cfs_fail_loc=159, val=0*** [ 2965.151842] Lustre: lustre-MDT0000: Client 254ddd0f-0ed3-425c-8e09-593d0afff5d8 (at 192.168.201.52@tcp) reconnecting [ 2966.741325] Lustre: *** cfs_fail_loc=159, val=0*** [ 2966.743229] Lustre: Skipped 5 previous similar messages [ 2983.036679] Lustre: lustre-MDT0000: Client 254ddd0f-0ed3-425c-8e09-593d0afff5d8 (at 192.168.201.52@tcp) reconnecting [ 3002.342746] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3002.346429] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3002.351000] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3002.355600] Lustre: Skipped 2 previous similar messages [ 3002.362493] Lustre: Skipped 3 previous similar messages [ 3006.113248] LustreError: 126907:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 3006.129161] LustreError: 126907:0:(obd_class.h:479:obd_check_dev()) Skipped 13 previous similar messages [ 3006.376322] Lustre: server umount lustre-OST0000 complete [ 3012.577988] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3012.579444] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3012.586125] Lustre: Skipped 1 previous similar message [ 3012.604868] Lustre: Skipped 1 previous similar message [ 3024.355522] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 3024.516291] LustreError: 127108:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup [ 3024.521972] LustreError: 127108:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 3024.701527] Lustre: server umount lustre-MDT0000 complete [ 3027.949341] LustreError: 125119:0:(ldlm_lib.c:1245:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3027.981574] LustreError: 125119:0:(ldlm_lib.c:1245:target_handle_connect()) Skipped 1 previous similar message [ 3029.470881] LustreError: 124372:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1761287339 with bad export cookie 18396878256719074884 [ 3029.471844] LustreError: MGC192.168.201.152@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3029.495573] LustreError: 124372:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 3030.177262] Lustre: server umount lustre-MDT0001 complete [ 3042.150392] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing unload_modules_local [ 3045.230696] Key type lgssc unregistered [ 3045.622835] LNet: 127844:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3045.638228] LNetError: 127844:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3045.677393] LNet: Removed LNI 192.168.201.152@tcp [ 3046.820605] Key type .llcrypt unregistered [ 3046.822447] Key type ._llcrypt unregistered [ 3072.041344] Key type ._llcrypt registered [ 3072.047415] Key type .llcrypt registered [ 3072.194325] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing load_modules_local [ 3073.058615] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3073.080871] alg: No test for adler32 (adler32-zlib) [ 3074.090361] Lustre: Lustre: Build Version: 2.16.59_49_g4265256 [ 3074.269180] LNet: Added LNI 192.168.201.152@tcp [8/256/0/180] [ 3075.944675] Key type lgssc registered [ 3077.216829] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3084.331863] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3084.357139] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3085.885956] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3089.858798] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 3095.016692] LustreError: 129239:0:(ldlm_lib.c:1245:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3095.027346] LustreError: 129239:0:(ldlm_lib.c:1245:target_handle_connect()) Skipped 2 previous similar messages [ 3097.594182] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3098.019822] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3102.248831] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 3107.150956] Lustre: DEBUG MARKER: oleg152-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 3110.860301] Lustre: DEBUG MARKER: oleg152-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 3118.323225] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3118.753532] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3125.464646] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 3130.089378] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:42 to 0x280000401:65) [ 3131.755213] Lustre: DEBUG MARKER: oleg152-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3141.261849] Lustre: *** cfs_fail_loc=159, val=0*** [ 3157.674656] Lustre: lustre-MDT0000: Client bf41f6c4-30b4-4de5-906b-06ce5c7e6a91 (at 192.168.201.52@tcp) reconnecting [ 3157.926697] Lustre: 130410:0:(osd_internal.h:1361:osd_trans_exec_op()) lustre-MDT0001: opcode 2: before 507 < left 525, rollback = 2 [ 3157.939097] Lustre: 130410:0:(osd_handler.c:2088:osd_trans_dump_creds()) create: 3/12/6, destroy: 0/0/0 [ 3157.948146] Lustre: 130410:0:(osd_handler.c:2095:osd_trans_dump_creds()) attr_set: 3/3/0, xattr_set: 6/525/0 [ 3157.955229] Lustre: 130410:0:(osd_handler.c:2105:osd_trans_dump_creds()) write: 7/100/0, punch: 0/0/0, quota 1/3/0 [ 3157.961142] Lustre: 130410:0:(osd_handler.c:2112:osd_trans_dump_creds()) insert: 7/135/3, delete: 0/0/0 [ 3157.965718] Lustre: 130410:0:(osd_handler.c:2119:osd_trans_dump_creds()) ref_add: 1/1/0, ref_del: 0/0/0 [ 3159.491976] Lustre: *** cfs_fail_loc=159, val=0*** [ 3175.071581] Lustre: lustre-MDT0001: Client bf41f6c4-30b4-4de5-906b-06ce5c7e6a91 (at 192.168.201.52@tcp) reconnecting [ 3176.599106] Lustre: *** cfs_fail_loc=159, val=0*** [ 3176.605561] Lustre: Skipped 3 previous similar messages [ 3191.921222] Lustre: lustre-MDT0001: Client bf41f6c4-30b4-4de5-906b-06ce5c7e6a91 (at 192.168.201.52@tcp) reconnecting [ 3208.046350] Lustre: 131967:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 3210.285489] Lustre: *** cfs_fail_loc=159, val=0*** [ 3210.287246] Lustre: Skipped 3 previous similar messages [ 3225.749541] Lustre: lustre-MDT0000: Client dd63f734-6100-40ba-8560-4c37a7a5a1ab (at 192.168.201.52@tcp) reconnecting [ 3227.510313] Lustre: *** cfs_fail_loc=159, val=0*** [ 3227.517233] Lustre: Skipped 13 previous similar messages [ 3243.164526] Lustre: lustre-MDT0000: Client dd63f734-6100-40ba-8560-4c37a7a5a1ab (at 192.168.201.52@tcp) reconnecting [ 3244.594649] Lustre: 132259:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 3250.443839] LustreError: 132356:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 3250.656627] Lustre: server umount lustre-OST0000 complete [ 3256.801457] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3256.817316] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3256.822070] LustreError: Skipped 1 previous similar message [ 3256.844409] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3258.343973] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3258.346032] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3263.462042] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3263.464829] Lustre: Skipped 1 previous similar message [ 3264.490354] LustreError: 132558:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup [ 3264.507849] LustreError: 132558:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 3264.574866] Lustre: server umount lustre-MDT0000 complete [ 3267.040207] Lustre: 128551:0:(client.c:2468:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1761287561/real 1761287561] req@000000005b5f4fab x1846843680037632/t0(0) o13->lustre-OST0000-osc-MDT0001@0@lo:7/4 lens 224/368 e 0 to 1 dl 1761287577 ref 1 fl Rpc:XQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 3267.065481] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3267.082221] LustreError: 132087:0:(ldlm_lib.c:1245:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3267.100184] LustreError: 132087:0:(ldlm_lib.c:1245:target_handle_connect()) Skipped 1 previous similar message [ 3267.387930] LustreError: 129219:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1761287577 with bad export cookie 17140142613435883625 [ 3267.390976] LustreError: MGC192.168.201.152@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3267.407768] LustreError: 129219:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 3267.504714] LustreError: 132761:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [ 3267.507918] LustreError: 132761:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 3267.702314] Lustre: server umount lustre-MDT0001 complete [ 3275.747895] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing unload_modules_local [ 3277.825635] Key type lgssc unregistered [ 3278.066640] LNet: 133292:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3278.080210] LNetError: 133292:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3278.102785] LNet: Removed LNI 192.168.201.152@tcp [ 3278.716503] Key type .llcrypt unregistered [ 3278.718360] Key type ._llcrypt unregistered [ 3298.770539] Key type ._llcrypt registered [ 3298.772179] Key type .llcrypt registered [ 3298.872967] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing load_modules_local [ 3299.364374] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3299.487119] alg: No test for adler32 (adler32-zlib) [ 3300.531971] Lustre: Lustre: Build Version: 2.16.59_49_g4265256 [ 3300.715678] LNet: Added LNI 192.168.201.152@tcp [8/256/0/180] [ 3302.384198] Key type lgssc registered [ 3303.233871] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3309.465030] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3309.476320] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3310.790573] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3313.737820] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 3320.290965] LustreError: 134684:0:(ldlm_lib.c:1245:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3320.312123] LustreError: 134684:0:(ldlm_lib.c:1245:target_handle_connect()) Skipped 2 previous similar messages [ 3320.797988] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3321.085810] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3324.554375] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 3330.438913] Lustre: DEBUG MARKER: oleg152-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 3334.248694] Lustre: DEBUG MARKER: oleg152-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 3341.549262] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3341.893452] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3346.044926] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:97) [ 3346.891383] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 3351.406339] Lustre: DEBUG MARKER: oleg152-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3355.260792] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:129) [ 3361.643763] Lustre: 136939:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 3363.726126] Lustre: 136987:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 3364.695887] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 in log params [ 3365.330817] Lustre: Setting parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=8 in log params [ 3367.440365] Lustre: 137134:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify lustre/mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 (mode = 0) failed: rc = -17 [ 3368.052219] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=7 in log params [ 3368.696788] Lustre: 137232:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 3372.004245] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3372.012248] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3372.020394] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3376.101600] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3376.105902] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3376.108508] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3376.119994] Lustre: Skipped 1 previous similar message [ 3377.276184] LustreError: 137329:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 3377.455398] Lustre: server umount lustre-OST0000 complete [ 3381.223889] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3381.234170] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3381.240603] Lustre: Skipped 1 previous similar message [ 3386.341222] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3386.360593] Lustre: Skipped 1 previous similar message [ 3386.625808] LustreError: 137538:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup [ 3386.653969] LustreError: 137538:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 3386.794823] Lustre: server umount lustre-MDT0000 complete [ 3390.798653] LustreError: 137036:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1761287700 with bad export cookie 4183929148910798393 [ 3390.800259] LustreError: MGC192.168.201.152@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3390.818359] LustreError: 137036:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 3391.031545] LustreError: 137739:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [ 3391.054410] LustreError: 137739:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 3391.326679] Lustre: server umount lustre-MDT0001 complete [ 3402.602966] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing unload_modules_local [ 3405.982437] Key type lgssc unregistered [ 3406.375607] LNet: 138272:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3406.387484] LNetError: 138272:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3406.410203] LNet: Removed LNI 192.168.201.152@tcp [ 3407.437817] Key type .llcrypt unregistered [ 3407.439354] Key type ._llcrypt unregistered [ 3429.735377] Key type ._llcrypt registered [ 3429.738321] Key type .llcrypt registered [ 3429.822053] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing load_modules_local [ 3430.677065] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3430.718065] alg: No test for adler32 (adler32-zlib) [ 3431.870838] Lustre: Lustre: Build Version: 2.16.59_49_g4265256 [ 3432.192809] LNet: Added LNI 192.168.201.152@tcp [8/256/0/180] [ 3433.864165] Key type lgssc registered [ 3435.118962] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3442.235893] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3442.284241] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3443.779346] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3447.164585] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 3452.898536] LustreError: 139667:0:(ldlm_lib.c:1245:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3452.931181] LustreError: 139667:0:(ldlm_lib.c:1245:target_handle_connect()) Skipped 2 previous similar messages [ 3455.091712] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3455.462626] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3458.897413] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 3463.730859] Lustre: DEBUG MARKER: oleg152-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 3467.012288] Lustre: DEBUG MARKER: oleg152-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 3472.462368] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3472.836922] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3478.232853] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 3479.998243] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:129) [ 3483.376174] Lustre: DEBUG MARKER: oleg152-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3488.192777] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:161) [ 3492.854765] Lustre: *** cfs_fail_loc=159, val=0*** [ 3509.428546] Lustre: lustre-MDT0000: Client 97703741-0797-4b34-945d-1f5c3c232265 (at 192.168.201.52@tcp) reconnecting [ 3526.116191] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3526.127445] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3526.136235] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3529.191383] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3529.194369] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3529.197735] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3529.217995] Lustre: Skipped 1 previous similar message [ 3531.921415] LustreError: 142049:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 3532.149463] Lustre: server umount lustre-OST0000 complete [ 3537.379362] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3537.386542] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3537.394375] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3539.426415] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3539.434679] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3539.510468] Lustre: Skipped 1 previous similar message [ 3541.552442] LustreError: 142250:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup [ 3541.554901] LustreError: 142250:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 3541.674965] Lustre: server umount lustre-MDT0000 complete [ 3544.551760] LustreError: 139668:0:(ldlm_lib.c:1245:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3544.564439] LustreError: 139668:0:(ldlm_lib.c:1245:target_handle_connect()) Skipped 1 previous similar message [ 3545.154960] LustreError: 139649:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1761287855 with bad export cookie 13236379189673609459 [ 3545.159198] LustreError: MGC192.168.201.152@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3545.180873] LustreError: 139649:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 3545.305915] LustreError: 142454:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [ 3545.310497] LustreError: 142454:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 3545.521243] Lustre: server umount lustre-MDT0001 complete [ 3554.802192] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing unload_modules_local [ 3557.024871] Key type lgssc unregistered [ 3557.275517] LNet: 142985:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3557.299019] LNetError: 142985:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3557.318699] LNet: Removed LNI 192.168.201.152@tcp [ 3558.068254] Key type .llcrypt unregistered [ 3558.071635] Key type ._llcrypt unregistered [ 3577.979376] Key type ._llcrypt registered [ 3577.984311] Key type .llcrypt registered [ 3578.141608] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing load_modules_local [ 3579.028244] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3579.078645] alg: No test for adler32 (adler32-zlib) [ 3580.165553] Lustre: Lustre: Build Version: 2.16.59_49_g4265256 [ 3580.377104] LNet: Added LNI 192.168.201.152@tcp [8/256/0/180] [ 3582.026228] Key type lgssc registered [ 3583.033680] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3589.627492] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3589.650831] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3591.139141] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3594.594345] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 3601.182978] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3601.383392] Lustre: lustre-MDT0001: Not available for connect from 0@lo (not set up) [ 3601.561169] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3604.679590] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 3609.742746] Lustre: DEBUG MARKER: oleg152-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 3613.076502] Lustre: DEBUG MARKER: oleg152-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 3619.968247] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3620.407288] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3626.553150] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:161) [ 3627.750226] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 3635.230417] Lustre: DEBUG MARKER: oleg152-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3636.800086] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:170 to 0x280000401:193) [ 3659.098624] Lustre: lustre-OST0000: evicting cdc1b225-e491-46c2-bb2b-e7eaa2c86863 (at 192.168.201.52@tcp) by administrative request [ 3659.105534] Lustre: Skipped 1 previous similar message [ 3687.905171] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3687.912642] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3687.913533] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3687.924915] Lustre: Skipped 1 previous similar message [ 3687.929860] Lustre: Skipped 1 previous similar message [ 3692.423842] LustreError: 147146:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 3692.573477] Lustre: server umount lustre-OST0000 complete [ 3698.146772] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3698.154593] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3698.166183] Lustre: Skipped 1 previous similar message [ 3698.179126] Lustre: Skipped 1 previous similar message [ 3701.848641] LustreError: 147348:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup [ 3701.852315] LustreError: 147348:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 3701.934745] Lustre: server umount lustre-MDT0000 complete [ 3703.270479] LustreError: 144361:0:(ldlm_lib.c:1245:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3703.279132] LustreError: 144361:0:(ldlm_lib.c:1245:target_handle_connect()) Skipped 3 previous similar messages [ 3704.641231] LustreError: 146728:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1761288014 with bad export cookie 9016540072258177813 [ 3704.645524] LustreError: MGC192.168.201.152@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3704.649877] LustreError: 146728:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 3704.780995] LustreError: 147549:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [ 3704.783951] LustreError: 147549:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 3704.947969] Lustre: server umount lustre-MDT0001 complete [ 3712.643484] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing unload_modules_local [ 3714.913327] Key type lgssc unregistered [ 3715.159938] LNet: 148082:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3715.169906] LNetError: 148082:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3715.183957] LNet: Removed LNI 192.168.201.152@tcp [ 3715.840096] Key type .llcrypt unregistered [ 3715.841687] Key type ._llcrypt unregistered [ 3785.153109] Key type ._llcrypt registered [ 3785.154630] Key type .llcrypt registered [ 3785.250780] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing load_modules_local [ 3786.219488] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3786.321140] alg: No test for adler32 (adler32-zlib) [ 3787.375806] Lustre: Lustre: Build Version: 2.16.59_49_g4265256 [ 3787.604533] LNet: Added LNI 192.168.201.152@tcp [8/256/0/180] [ 3789.288146] Key type lgssc registered [ 3790.352959] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3797.178949] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3797.197182] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3798.606650] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3801.720880] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 3803.617166] LustreError: 150557:0:(ldlm_lib.c:1245:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3803.636536] LustreError: 150557:0:(ldlm_lib.c:1245:target_handle_connect()) Skipped 2 previous similar messages [ 3807.971769] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3808.238784] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3811.322897] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 3815.690086] Lustre: DEBUG MARKER: oleg152-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 3818.447056] Lustre: DEBUG MARKER: oleg152-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 3823.289898] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3823.602434] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3824.645957] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:195 to 0x280000401:225) [ 3828.503638] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 3833.599427] Lustre: DEBUG MARKER: oleg152-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3833.852290] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:193) [ 3844.067358] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3844.070527] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3844.081379] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3844.102415] Lustre: Skipped 1 previous similar message [ 3847.092109] LustreError: 152814:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 3847.269543] Lustre: server umount lustre-OST0000 complete [ 3854.305825] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3854.311734] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3854.311743] Lustre: Skipped 1 previous similar message [ 3854.315610] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3854.315620] Lustre: Skipped 1 previous similar message [ 3856.411515] LustreError: 153015:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup [ 3856.427113] LustreError: 153015:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 3856.586828] Lustre: server umount lustre-MDT0000 complete [ 3859.429229] LustreError: 151272:0:(ldlm_lib.c:1245:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3859.462322] LustreError: 151272:0:(ldlm_lib.c:1245:target_handle_connect()) Skipped 1 previous similar message [ 3859.871555] LustreError: 150536:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1761288169 with bad export cookie 9036444890224309319 [ 3859.878061] LustreError: MGC192.168.201.152@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3859.881441] LustreError: 150536:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 3860.045209] LustreError: 153218:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [ 3860.047978] LustreError: 153218:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 3860.268927] Lustre: server umount lustre-MDT0001 complete [ 3868.831555] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing unload_modules_local [ 3870.766624] Key type lgssc unregistered [ 3870.999601] LNet: 153749:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3871.006519] LNetError: 153749:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3871.018676] LNet: Removed LNI 192.168.201.152@tcp [ 3871.589703] Key type .llcrypt unregistered [ 3871.591820] Key type ._llcrypt unregistered [ 3887.630049] Key type ._llcrypt registered [ 3887.631839] Key type .llcrypt registered [ 3887.696903] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing load_modules_local [ 3888.208159] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3888.233564] alg: No test for adler32 (adler32-zlib) [ 3889.159294] Lustre: Lustre: Build Version: 2.16.59_49_g4265256 [ 3889.295125] LNet: Added LNI 192.168.201.152@tcp [8/256/0/180] [ 3890.912208] Key type lgssc registered [ 3891.521166] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3896.445547] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3901.827642] Lustre: DEBUG MARKER: == conf-sanity test 100: check lshowmount lists MGS, MDT, OST and 0@lo ========================================================== 02:43:31 (1761288211) [ 3913.704710] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_hostid [ 3919.962752] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing load_modules_local [ 3926.166988] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3931.025754] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3935.649540] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3940.116794] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3945.088551] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3945.154486] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3945.171377] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3946.360912] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3946.393617] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3946.451740] Lustre: lustre-MDT0000: new disk, initializing [ 3946.512782] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3946.540185] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3949.518738] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 3956.772271] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3956.815265] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3956.871425] Lustre: 157948:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 3956.931284] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 3956.934822] Lustre: Skipped 1 previous similar message [ 3957.004287] Lustre: lustre-MDT0001: new disk, initializing [ 3957.082616] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3957.115995] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 3957.123689] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3959.552892] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 3964.503510] Lustre: DEBUG MARKER: oleg152-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 3967.004308] Lustre: DEBUG MARKER: oleg152-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 3970.909081] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3970.946626] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3971.082635] Lustre: lustre-OST0000: new disk, initializing [ 3971.086845] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3971.133500] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3974.185737] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 3977.233380] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 3977.237664] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 3977.316970] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3978.557636] Lustre: DEBUG MARKER: oleg152-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3987.426217] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3987.428390] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3987.440190] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3987.445143] Lustre: Skipped 1 previous similar message [ 3991.488389] LustreError: 160539:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 3991.622369] Lustre: server umount lustre-OST0000 complete [ 3997.665631] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3997.667811] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3997.672952] Lustre: Skipped 2 previous similar messages [ 4000.343835] LustreError: 160740:0:(obd_class.h:479:obd_check_dev()) Device 22 not setup [ 4000.349868] LustreError: 160740:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 4002.406868] LustreError: 160740:0:(obd_class.h:479:obd_check_dev()) Device 8 not setup [ 4002.409485] LustreError: 160740:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 4002.432739] Lustre: server umount lustre-MDT0000 complete [ 4004.698565] LustreError: 158730:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1761288314 with bad export cookie 3793967520019746636 [ 4004.699398] LustreError: MGC192.168.201.152@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4004.710629] LustreError: 158730:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4004.823991] LustreError: 160942:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 4004.829383] LustreError: 160942:0:(obd_class.h:479:obd_check_dev()) Skipped 1 previous similar message [ 4004.940145] Lustre: server umount lustre-MDT0001 complete [ 4010.654843] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing unload_modules_local [ 4012.164091] Key type lgssc unregistered [ 4012.331826] LNet: 161474:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4012.335548] LNetError: 161474:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4012.347475] LNet: Removed LNI 192.168.201.152@tcp [ 4012.826578] Key type .llcrypt unregistered [ 4012.828369] Key type ._llcrypt unregistered [ 4027.265138] Key type ._llcrypt registered [ 4027.269800] Key type .llcrypt registered [ 4027.370188] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing load_modules_local [ 4027.961076] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4028.023956] alg: No test for adler32 (adler32-zlib) [ 4028.955754] Lustre: Lustre: Build Version: 2.16.59_49_g4265256 [ 4029.134131] LNet: Added LNI 192.168.201.152@tcp [8/256/0/180] [ 4030.768378] Key type lgssc registered [ 4031.390044] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4036.050585] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4036.070616] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4037.375333] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4039.557850] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 4044.650031] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4044.857180] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4047.183772] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 4050.254711] Lustre: DEBUG MARKER: oleg152-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 4052.580763] Lustre: DEBUG MARKER: oleg152-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 4056.784854] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4056.979790] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4060.239466] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 4062.142627] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:3 to 0x280000401:33) [ 4063.892861] Lustre: DEBUG MARKER: oleg152-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4072.077650] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4072.090920] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4072.110417] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4072.135570] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4072.142464] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4072.918602] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4072.932091] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4072.942932] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4072.949774] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4072.957693] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4073.889736] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4074.574704] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4074.578980] Lustre: Skipped 1 previous similar message [ 4074.590620] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4074.597193] Lustre: Skipped 1 previous similar message [ 4074.600597] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4074.604553] Lustre: Skipped 1 previous similar message [ 4074.607402] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4074.627326] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4074.630300] Lustre: Skipped 1 previous similar message [ 4075.431232] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4076.343385] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4077.218471] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4077.227336] Lustre: Skipped 2 previous similar messages [ 4077.293314] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4077.299322] Lustre: Skipped 2 previous similar messages [ 4077.304603] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4077.308330] Lustre: Skipped 2 previous similar messages [ 4077.312588] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4077.320137] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4077.328864] Lustre: Skipped 2 previous similar messages [ 4078.476382] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4080.012854] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4081.185804] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4083.165319] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4083.183687] Lustre: Skipped 3 previous similar messages [ 4083.211243] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4083.236381] Lustre: Skipped 3 previous similar messages [ 4083.244720] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4083.257181] Lustre: Skipped 3 previous similar messages [ 4083.264924] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4083.283131] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4083.304265] Lustre: Skipped 3 previous similar messages [ 4084.948706] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4086.916837] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4088.283723] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4090.024907] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4091.761910] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4091.778809] Lustre: Skipped 4 previous similar messages [ 4091.802736] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4091.808086] Lustre: Skipped 4 previous similar messages [ 4091.816714] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4091.819582] Lustre: Skipped 4 previous similar messages [ 4091.826620] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4091.866350] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4091.869085] Lustre: Skipped 4 previous similar messages [ 4093.235554] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4094.561889] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4095.802492] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4097.374598] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4098.962827] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4100.625650] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4102.120910] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4103.873363] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4105.650204] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4107.381659] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4109.230748] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4109.247540] Lustre: Skipped 10 previous similar messages [ 4109.299782] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4109.318933] Lustre: Skipped 10 previous similar messages [ 4109.332351] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4109.335742] Lustre: Skipped 10 previous similar messages [ 4109.348811] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4109.369229] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4109.393933] Lustre: Skipped 10 previous similar messages [ 4111.050803] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4112.724908] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4113.871919] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4115.412234] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4116.935602] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4118.239961] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4119.282691] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4120.453242] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4121.551604] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4122.896102] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4123.667851] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4124.871639] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4126.085219] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4127.073778] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4127.882691] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4128.658772] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4129.533408] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4130.504558] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4131.580786] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4132.802721] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4133.613162] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4134.698597] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4135.545988] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4136.390163] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4137.151052] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4138.189768] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4138.943090] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4139.853501] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4140.747022] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4141.651449] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4141.657559] Lustre: Skipped 29 previous similar messages [ 4141.673292] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4141.679297] Lustre: Skipped 29 previous similar messages [ 4141.682243] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4141.686195] Lustre: Skipped 29 previous similar messages [ 4141.688582] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4141.697717] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4141.700041] Lustre: Skipped 29 previous similar messages [ 4142.486270] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4143.471722] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4144.325967] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4145.456549] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4146.414529] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4147.359052] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4148.238655] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4149.326170] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4150.293908] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4151.146577] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4152.174780] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4153.032479] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4153.891542] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4154.701566] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4155.610777] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4156.473893] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4157.433726] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4158.298303] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4159.071390] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4159.999240] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4160.862877] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4161.607583] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4162.452416] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4163.331971] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4164.091422] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4164.890879] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4165.710518] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4166.478295] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4167.297646] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4168.021721] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4168.834920] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4169.438199] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4170.185946] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4171.014672] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4171.737122] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4172.480589] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4173.256552] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4174.020566] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4174.744858] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4175.499088] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4176.316175] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4177.034506] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4177.772826] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4178.569329] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4179.325835] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4180.010158] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4180.711919] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4181.408530] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4182.159276] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4182.828193] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4183.605772] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4184.384832] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4185.197281] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4185.875925] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4186.656217] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4187.543477] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4188.280895] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4189.084036] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4190.002123] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4190.813559] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4191.633826] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4192.679485] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4193.528518] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4194.445944] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4195.476086] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4196.327996] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4197.118497] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4197.814785] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4198.553555] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4199.356069] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4200.128941] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4200.862255] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4201.682341] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4202.414910] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4203.139718] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4203.844255] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4204.551844] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4205.250197] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4206.055162] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4206.068202] Lustre: Skipped 78 previous similar messages [ 4206.095561] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4206.103357] Lustre: Skipped 78 previous similar messages [ 4206.111726] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4206.115142] Lustre: Skipped 78 previous similar messages [ 4206.122568] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4206.152508] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4206.155609] Lustre: Skipped 78 previous similar messages [ 4206.919642] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4207.757783] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4208.532763] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4209.286578] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4210.104858] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4210.836767] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4211.497183] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4212.246754] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4213.025286] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4213.965265] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4214.810660] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4215.636217] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4216.378658] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4217.073176] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4217.736325] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4218.362883] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4219.173979] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4219.896240] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4220.590133] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4221.398479] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4222.157868] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4222.896292] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4223.563375] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4224.290161] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4225.168271] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4226.045136] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4226.871686] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4227.583256] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4228.465776] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4229.232580] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4230.159610] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4230.898529] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4231.710439] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4232.434605] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4233.238882] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4233.965871] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4234.709092] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4235.435888] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4236.208851] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4236.892536] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4237.612303] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4238.330189] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4239.082054] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4239.860561] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4240.583917] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4241.226069] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4241.949908] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4242.594652] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4243.355331] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4244.025498] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4244.841298] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4245.515656] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4246.231238] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4246.939287] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4247.705665] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4248.422109] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4249.161513] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4249.886697] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4250.655765] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4251.492285] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4252.221235] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4253.077765] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4253.822783] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4254.595527] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4255.328741] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4256.154764] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4256.977081] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4257.815650] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4258.538671] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4259.316776] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4260.225960] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4260.946370] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4261.672588] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4262.583233] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4263.255872] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4264.103996] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4264.748571] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4265.419419] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4266.052841] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4266.674924] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4267.430206] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4268.051884] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4268.737830] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4269.568915] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4270.373521] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4271.213712] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4272.033189] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4272.823362] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4273.574333] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4274.493690] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4275.303652] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4276.131468] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4277.031622] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4277.942471] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4278.691924] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4279.491267] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4280.454838] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4281.240468] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4282.043506] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4282.847066] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4283.665751] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4284.464324] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4285.267626] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4286.073947] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4286.898736] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4287.665679] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4288.419955] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4289.085798] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4289.899268] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4290.601595] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4291.430659] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4292.037527] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4292.776269] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4293.608789] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4294.503472] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4295.291804] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4296.180964] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4297.040931] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4297.945378] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4298.849686] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4299.684898] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4300.545342] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4301.351142] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4302.151917] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4302.977733] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4303.639705] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4304.397046] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4305.054230] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4305.877644] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4306.672667] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4307.451637] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4308.148776] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4308.933313] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4309.876475] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4310.851699] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4311.569714] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4312.465775] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4313.374513] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4314.220728] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4315.103060] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4316.265559] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4317.185309] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4318.090166] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4318.906604] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4319.795792] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4320.552272] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4321.412498] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4322.147731] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4322.990874] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4323.977655] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4324.956091] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4325.947136] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4326.861840] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4327.801053] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4328.909544] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4330.018297] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4330.863618] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4331.733080] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4332.547390] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4333.475220] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4334.339471] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4334.343703] Lustre: Skipped 160 previous similar messages [ 4334.363830] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4334.369347] Lustre: Skipped 160 previous similar messages [ 4334.371964] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4334.382773] Lustre: Skipped 160 previous similar messages [ 4334.387592] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4334.398788] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4334.416710] Lustre: Skipped 160 previous similar messages [ 4335.120193] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4335.972727] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4336.819197] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4337.573620] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4338.230127] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4338.940822] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4339.678771] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4340.286835] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4340.957073] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4341.701502] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4342.353416] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4343.019326] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4343.756585] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4344.609648] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4345.460827] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4346.311747] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4347.009899] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4347.777892] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4348.603406] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4349.375163] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4350.271903] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4351.174202] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4351.995576] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4352.752362] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4353.460088] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4354.256437] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4355.030916] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4355.793411] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4356.515772] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4357.225789] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4357.973631] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4358.705382] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4359.396821] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4360.143645] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4360.926340] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4361.656463] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4362.461226] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4363.297347] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4364.015650] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4364.766891] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4365.493797] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4366.210927] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4366.857448] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4367.617491] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4368.346884] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4369.170462] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4370.050552] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4370.845796] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4371.604063] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4372.346984] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4373.042857] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4373.876455] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4374.489971] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4375.286847] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4375.897877] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4376.583807] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4377.204334] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4377.840564] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4378.475153] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4379.134856] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4379.794812] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4380.450087] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4381.152770] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4382.075995] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4382.830375] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4383.675222] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4384.400960] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4385.173061] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4385.903230] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4386.662775] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4387.400158] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4388.181789] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4389.018960] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4389.819977] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4390.523775] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4391.281829] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4392.068686] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4392.690806] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4393.427373] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4394.100809] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4394.790034] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4395.358770] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4396.035173] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4396.665982] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4397.327183] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4397.980037] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4398.726779] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4399.295302] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4399.965445] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4400.525382] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4401.170068] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4401.939431] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4402.790719] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4403.475493] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4404.182564] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4404.845419] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4405.468293] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4406.155989] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4406.760048] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4407.311096] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4407.914663] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4408.472488] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4409.015479] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4409.610854] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4410.152958] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4410.712074] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4411.342334] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4411.993784] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4412.620320] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4413.162484] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4413.731511] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4414.301355] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4414.894590] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4415.526346] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4416.144512] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4416.698455] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4417.280619] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4417.906446] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4418.477971] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4419.023243] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4419.539724] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4420.112521] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4420.638570] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4421.217382] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4421.787218] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4422.331283] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4422.850533] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4423.380222] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4423.922948] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4424.466799] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4424.993280] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4425.497567] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4426.022671] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4426.509973] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4426.985072] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4427.450693] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4427.917824] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4428.376086] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4428.815758] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4429.261799] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4429.723455] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4430.183860] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4430.630288] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4431.131745] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4431.629650] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4432.152862] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4432.665493] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4433.181082] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4433.719441] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4434.232149] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4434.778316] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4435.356355] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4435.886498] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4436.434294] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4436.976581] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4437.576619] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4438.134871] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4438.677976] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4439.350431] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4439.874051] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4440.424873] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4440.903552] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4441.481793] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4442.071664] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4442.634060] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4443.209122] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4443.775535] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4444.362312] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4444.916960] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4445.417276] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4445.952038] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4446.491572] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4446.996370] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4447.483817] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4448.047772] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4448.579969] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4449.104280] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4449.613477] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4450.043176] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4450.468568] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4450.903570] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4451.326606] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4451.783286] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4452.265566] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4452.687314] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4453.101932] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4453.529034] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4453.946536] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4454.344489] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4454.727745] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4455.163229] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4455.562580] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4455.985209] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4456.465398] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4456.880152] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4457.303328] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4457.728614] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4458.141227] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4458.546885] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4458.966170] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4459.400170] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4459.894173] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4460.359168] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4460.811632] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4461.253290] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4461.692308] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4462.133794] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4462.625188] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4463.086110] LustreError: 162867:0:(osp_precreate.c:651:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 4463.091124] LustreError: 162867:0:(osp_precreate.c:1355:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 4463.093069] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4463.528485] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4463.980451] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4464.522447] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4465.017684] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4465.496384] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4465.929939] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4466.372093] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4466.816188] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4467.232232] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4467.659492] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4468.072386] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4468.468232] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4468.857585] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4469.228145] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4469.624248] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4470.023792] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4470.431628] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4470.827517] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4471.213078] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4471.618616] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4472.056059] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4472.455967] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4472.865900] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4473.264199] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4473.741723] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4474.178737] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4474.584279] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4474.988588] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4475.385855] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4475.775330] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4476.184571] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4476.588144] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4477.026040] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4477.471766] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4477.914250] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4478.338701] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4478.817426] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4479.314571] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4479.786070] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4480.236367] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4480.678990] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4481.176431] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4481.635644] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4482.108281] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4482.567486] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4483.030467] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4483.519925] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4484.005075] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4484.471112] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4484.964354] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4485.467614] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4485.955661] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4486.439763] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4486.880476] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4487.363422] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4487.836776] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4488.303364] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4488.766910] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4489.322989] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4489.775771] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4490.218311] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4490.666203] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4491.103079] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4491.504075] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4491.911085] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4492.299447] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4492.706682] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4493.133295] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4493.553263] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4493.952206] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4494.362826] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4494.784178] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4495.226743] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4495.652764] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4496.079610] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4496.475454] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4496.877197] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4497.276351] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4497.695489] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4498.119070] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4498.546420] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4498.991678] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4499.443521] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4499.903885] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4500.369852] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4500.787176] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4501.212160] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4501.675670] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4502.152170] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4502.598823] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4503.067248] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4503.527927] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4503.937466] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4504.348144] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4504.778470] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4505.215220] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4505.647818] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4506.074173] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4506.511084] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4506.909284] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4507.309768] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4507.704498] LustreError: 162867:0:(osp_precreate.c:651:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 4507.709279] LustreError: 162867:0:(osp_precreate.c:1355:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 4507.710279] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4508.101314] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4508.512366] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4508.956061] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4509.353421] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4509.765770] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4510.208193] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4510.662984] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4511.128462] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4511.616161] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4512.064492] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4512.517303] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4513.000831] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4513.454953] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4513.921381] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4514.389710] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4514.851524] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4515.306803] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4515.757506] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4516.213192] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4516.638977] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4517.116023] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4517.558055] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4518.016375] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4518.466899] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4518.894561] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4519.310390] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4519.749433] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4520.136429] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4520.567234] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4520.990372] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4521.421641] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4521.854114] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4522.290652] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4522.695128] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4523.130772] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4523.565219] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4523.972400] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4524.385301] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4524.819522] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4525.274443] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4525.696584] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4526.140264] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4526.586706] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4527.083539] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4527.575101] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4528.057361] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4528.507103] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4528.960685] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4529.364478] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4529.781501] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4530.224975] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4530.669647] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4531.123402] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4531.586924] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4532.006763] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4532.428325] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4532.878193] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4533.289809] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4533.739838] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4534.200079] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4534.636202] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4535.083456] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4535.610208] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4536.071428] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4536.499094] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4536.925332] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4537.364541] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4537.763991] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4538.211317] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4538.631167] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4539.070534] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4539.525748] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4539.995651] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4540.423194] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4540.867256] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4541.325836] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4541.748588] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4542.190611] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4542.667576] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4543.110872] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4543.589870] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4544.040608] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4544.491778] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4544.915979] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4545.356413] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4545.802696] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4546.265398] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4546.695995] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4547.123411] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4547.584044] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4548.030488] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4548.472429] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4548.920154] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4549.339381] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4549.757057] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4550.180841] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4550.582297] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4551.015321] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4551.478215] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4551.924268] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4552.336655] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4552.768186] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4553.191627] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4553.639249] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4554.090375] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4554.504214] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4554.920282] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4555.345753] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4555.825918] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4556.275308] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4556.715350] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4557.113696] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4557.548455] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4558.000980] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4558.430606] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4558.899849] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4559.368431] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4559.832780] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4560.276281] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4560.756486] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4561.235216] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4561.736811] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4562.225093] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4562.685247] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4563.153123] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4563.605742] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4564.056322] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4564.517586] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4564.973864] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4565.426947] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4565.890767] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4566.427213] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4566.944468] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4567.458629] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4567.924206] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4568.415994] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4568.909966] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4569.384210] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4569.879834] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4570.332868] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4570.804319] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4571.350953] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4571.826232] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4572.296475] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4572.764597] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4573.226059] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4573.711226] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4574.140120] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4574.580686] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4575.020434] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4575.490820] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4575.920649] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4576.428561] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4576.894675] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4577.395853] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4577.854132] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4578.308323] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4578.763759] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4579.197556] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4579.629708] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4580.060839] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4580.539813] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4581.007940] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4581.482264] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4581.919332] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4582.329068] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4582.826066] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4583.245615] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4583.669996] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4584.098072] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4584.534016] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4584.971552] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4585.437318] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4585.913765] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4586.403905] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4586.891219] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4587.339766] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4587.761791] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4588.240129] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4588.677188] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4589.086278] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4589.495913] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4589.932367] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4590.355848] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4590.358613] Lustre: Skipped 494 previous similar messages [ 4590.366836] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4590.372584] Lustre: Skipped 494 previous similar messages [ 4590.375316] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4590.378558] Lustre: Skipped 494 previous similar messages [ 4590.380869] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4590.771919] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4590.777497] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4590.780345] Lustre: Skipped 495 previous similar messages [ 4591.186317] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4591.616639] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4592.043440] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4592.487525] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4592.946422] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4593.406333] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4593.867156] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4594.329038] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4594.785668] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4595.233983] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4595.693302] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4596.136758] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4596.584362] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4597.031910] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4597.463504] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4597.884556] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4598.280359] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4598.718160] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4599.124177] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4599.550302] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4599.961831] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4600.377632] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4600.803960] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4601.253332] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4601.684785] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4602.136398] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4602.585946] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4603.090444] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4603.580578] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4604.052866] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4604.586411] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4605.115660] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4605.625146] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4606.087520] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4606.566988] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4606.973547] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4607.391781] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4607.838548] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4608.278471] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4608.698281] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4609.159393] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4609.596121] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4609.995953] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4610.380497] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4610.769937] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4611.165150] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4611.569672] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4611.992564] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4612.433049] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4612.827961] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4613.246463] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4613.706475] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4614.111537] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4614.476779] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4614.887757] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4615.294946] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4615.687645] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4616.097064] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4616.494424] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4616.899333] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4617.346302] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4617.740892] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4618.119038] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4618.519079] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4618.897751] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4619.262186] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4619.652589] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4620.057970] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4620.463440] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4620.872897] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4621.255270] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4621.651534] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4622.038254] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4622.449072] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4622.836756] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4623.220594] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4623.609372] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4624.002151] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4624.374059] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4624.772894] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4625.181852] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4625.591494] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4626.038913] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4626.490310] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4626.920306] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4627.339412] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4627.765062] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4628.190050] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4628.586320] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4628.993333] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4629.409410] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4629.858691] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4630.307790] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4630.780439] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4631.230479] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4631.729188] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4632.175811] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4632.656801] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4633.080443] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4633.511528] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4633.954387] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4634.434550] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4634.894862] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4635.330347] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4635.732125] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4636.164403] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4636.610911] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4637.058294] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4637.517855] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4638.004665] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4638.456407] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4638.911469] LustreError: 162867:0:(osp_precreate.c:651:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 4638.916968] LustreError: 162867:0:(osp_precreate.c:1355:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 4638.922201] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4639.397699] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4639.847139] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4640.270521] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4640.726298] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4641.164332] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4641.631749] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4642.053025] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4642.480410] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4642.914586] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4643.335384] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4643.771192] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4644.197111] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4644.616668] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4645.032702] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4645.448822] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4645.869046] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4646.265340] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4646.726301] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4647.205632] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4647.650281] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4648.134531] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4648.662993] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4649.154647] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4649.677543] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4650.150270] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4650.662976] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4651.141265] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4651.630625] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4652.126821] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4652.659723] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4653.181805] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4653.672539] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4654.175962] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4654.647435] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4655.103861] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4655.579911] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4656.067807] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4656.510336] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4656.995362] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4657.479237] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4657.903585] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4658.370417] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4658.788404] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4659.243022] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4659.734517] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4660.215340] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4660.679719] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4661.145880] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4661.618261] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4662.093658] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4662.526902] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4662.995626] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4663.454328] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4663.912042] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4664.397503] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4664.823920] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4665.283051] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4665.752062] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4666.182680] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4666.627937] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4667.077711] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4667.495425] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4667.935922] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4668.388541] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4668.841974] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4669.280464] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4669.695798] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4670.112827] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4670.548909] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4670.986035] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4671.430089] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4671.905132] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4672.351192] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4672.809357] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4673.319310] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4673.837123] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4674.366041] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4674.922018] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4675.462214] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4676.023807] LustreError: 162867:0:(osp_precreate.c:651:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 4676.031815] LustreError: 162867:0:(osp_precreate.c:1355:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 4676.041591] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4676.662972] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4677.246430] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4677.774031] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4678.225871] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4678.667761] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4679.139907] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4679.612695] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4680.099560] LustreError: 162867:0:(osp_precreate.c:651:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 4680.104341] LustreError: 162867:0:(osp_precreate.c:1355:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 4680.108663] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4680.579855] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4681.076832] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4681.586249] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4682.032774] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4682.492946] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4682.949968] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4683.405703] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4990.002676] LustreError: 164448:0:(ofd_dev.c:1780:ofd_create_hdl()) lustre-OST0000: unable to precreate: rc = -28 [ 4990.007734] LustreError: 162867:0:(osp_precreate.c:651:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -28 [ 5195.745414] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 5195.751058] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5195.756162] Lustre: Skipped 209 previous similar messages [ 5195.757391] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5195.760862] Lustre: Skipped 1 previous similar message [ 5196.772096] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5196.774779] Lustre: Skipped 1 previous similar message [ 5199.026083] LustreError: 216699:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 5199.140424] Lustre: server umount lustre-OST0000 complete [ 5201.891489] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 5201.892358] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5201.895335] LustreError: Skipped 8 previous similar messages [ 5201.898080] Lustre: Skipped 1 previous similar message [ 5207.010303] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5207.013573] Lustre: Skipped 1 previous similar message [ 5212.130650] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5212.133655] Lustre: Skipped 1 previous similar message [ 5215.200260] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 5215.254886] LustreError: 216902:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup [ 5215.257766] LustreError: 216902:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 5215.396154] Lustre: server umount lustre-MDT0000 complete [ 5216.978657] LustreError: 216299:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1761289527 with bad export cookie 16904678640559261865 [ 5216.982032] LustreError: MGC192.168.201.152@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5216.983918] LustreError: 216299:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 5217.059388] LustreError: 217104:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [ 5217.061474] LustreError: 217104:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 5217.162628] Lustre: server umount lustre-MDT0001 complete [ 5221.119765] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing unload_modules_local [ 5222.355481] Key type lgssc unregistered [ 5222.501552] LNet: 217635:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5222.505241] LNetError: 217635:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5222.512577] LNet: Removed LNI 192.168.201.152@tcp [ 5222.879069] Key type .llcrypt unregistered [ 5222.880624] Key type ._llcrypt unregistered [ 5233.992237] Key type ._llcrypt registered [ 5233.993637] Key type .llcrypt registered [ 5234.038207] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing load_modules_local [ 5234.556747] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5234.575626] alg: No test for adler32 (adler32-zlib) [ 5235.473689] Lustre: Lustre: Build Version: 2.16.59_49_g4265256 [ 5235.591722] LNet: Added LNI 192.168.201.152@tcp [8/256/0/180] [ 5237.200200] Key type lgssc registered [ 5237.726600] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5241.134957] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5241.145461] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5242.362671] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5244.093379] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 5247.770087] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5247.921107] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5249.625650] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 5252.009434] Lustre: DEBUG MARKER: oleg152-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 5253.617477] Lustre: DEBUG MARKER: oleg152-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 5256.683377] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5256.864948] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5259.428284] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 5261.790744] Lustre: DEBUG MARKER: oleg152-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 5267.121609] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49666 to 0x280000401:49697) [ 5270.024280] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5270.843872] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5270.852572] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5270.856827] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5270.864881] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5270.865273] LustreError: 219027:0:(osp_precreate.c:1263:osp_precreate_thread()) cfs_fail_timeout id 2107 sleeping for 20000ms [ 5272.548808] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5272.551480] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5272.558028] Lustre: Skipped 1 previous similar message [ 5273.184711] Lustre: lustre-OST0000: Not available for connect from 192.168.201.52@tcp (stopping) [ 5273.187892] Lustre: Skipped 1 previous similar message [ 5273.795255] LustreError: 221409:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 5273.893050] Lustre: server umount lustre-OST0000 complete [ 5290.944123] LustreError: 219027:0:(osp_precreate.c:1263:osp_precreate_thread()) cfs_fail_timeout id 2107 awake [ 5303.312260] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5303.457608] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5305.847659] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 5308.049563] Lustre: DEBUG MARKER: oleg152-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 5312.680928] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5312.686149] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5312.686163] LustreError: lustre-OST0000-osc-MDT0001: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5313.876342] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid in FULL state after 5 sec [ 5315.416755] Lustre: DEBUG MARKER: oleg152-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff940487a46000.ost_server_uuid 50 [ 5316.077867] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff940487a46000.ost_server_uuid in FULL state after 0 sec [ 5318.116870] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5318.118296] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 5318.121931] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5318.126201] Lustre: Skipped 1 previous similar message [ 5323.235575] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5323.239196] Lustre: Skipped 1 previous similar message [ 5323.264231] LustreError: 222731:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 5323.267769] LustreError: 222731:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 5323.362658] Lustre: server umount lustre-OST0000 complete [ 5328.354347] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5328.355436] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5328.358136] Lustre: Skipped 2 previous similar messages [ 5331.142817] LustreError: 222933:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup [ 5331.144957] LustreError: 222933:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 5331.179559] Lustre: server umount lustre-MDT0000 complete [ 5332.750458] LustreError: 219727:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1761289642 with bad export cookie 12159047690571102691 [ 5332.752979] LustreError: MGC192.168.201.152@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5332.755623] LustreError: 219727:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 5332.936312] Lustre: server umount lustre-MDT0001 complete [ 5336.961324] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing unload_modules_local [ 5338.170986] Key type lgssc unregistered [ 5338.303397] LNet: 223666:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5338.306738] LNetError: 223666:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5338.318455] LNet: Removed LNI 192.168.201.152@tcp [ 5338.708255] Key type .llcrypt unregistered [ 5338.710087] Key type ._llcrypt unregistered [ 5352.340380] Key type ._llcrypt registered [ 5352.341690] Key type .llcrypt registered [ 5352.394227] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing load_modules_local [ 5352.884949] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5352.906523] alg: No test for adler32 (adler32-zlib) [ 5353.787444] Lustre: Lustre: Build Version: 2.16.59_49_g4265256 [ 5353.895143] LNet: Added LNI 192.168.201.152@tcp [8/256/0/180] [ 5355.488299] Key type lgssc registered [ 5355.995687] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5359.714739] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5359.724359] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5360.969614] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5362.535261] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 5366.104561] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5366.293742] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5368.012299] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 5369.305696] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 5371.958118] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5372.109286] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5373.166770] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49699 to 0x280000401:49729) [ 5374.276959] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 5377.641078] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5377.664727] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5377.736069] Lustre: lustre-OST0001: new disk, initializing [ 5377.738644] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 5377.774577] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 5379.851510] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 5381.831831] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 5381.836633] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 5381.870799] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 5385.701268] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 5392.733588] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 5407.201974] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 5407.205126] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5407.210754] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5407.715486] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5407.719464] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5407.719651] Lustre: Skipped 2 previous similar messages [ 5407.722156] Lustre: Skipped 1 previous similar message [ 5408.001417] LustreError: 228711:0:(obd_class.h:479:obd_check_dev()) Device 28 not setup [ 5408.042962] Lustre: server umount lustre-MDT0000 complete [ 5409.549400] LustreError: 226455:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1761289719 with bad export cookie 18262947275277123127 [ 5409.553213] LustreError: MGC192.168.201.152@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5409.553273] LustreError: 226455:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 5409.828471] LustreError: 228912:0:(obd_class.h:479:obd_check_dev()) Device 27 not setup [ 5409.831292] LustreError: 228912:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 5409.932736] Lustre: server umount lustre-MDT0001 complete [ 5421.056113] LustreError: 229112:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 5421.059380] LustreError: 229112:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 5421.085340] Lustre: server umount lustre-OST0000 complete [ 5432.836874] LustreError: 229314:0:(obd_class.h:479:obd_check_dev()) Device 2 not setup [ 5432.842189] LustreError: 229314:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 5432.946763] Lustre: server umount lustre-OST0001 complete [ 5434.902572] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro [ 5434.975194] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5435.872837] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro [ 5436.002218] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5436.892849] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: ,errors=remount-ro [ 5436.990638] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5438.396411] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: ,errors=remount-ro [ 5438.555787] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5455.573796] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing load_modules_local [ 5467.712894] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5468.506326] LustreError: 230725:0:(ldlm_lib.c:1245:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5468.531628] LustreError: 230725:0:(ldlm_lib.c:1245:target_handle_connect()) Skipped 2 previous similar messages [ 5468.621957] Lustre: mylustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5473.347373] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 5473.777566] LustreError: 230725:0:(ldlm_lib.c:1245:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5477.890514] LustreError: 230725:0:(ldlm_lib.c:1245:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5482.985284] LustreError: 230725:0:(ldlm_lib.c:1245:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5483.195980] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5483.642084] Lustre: mylustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5488.601223] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 5491.812705] Lustre: 231837:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 5497.886424] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5502.546132] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 5508.465308] LustreError: 232194:0:(ldlm_lib.c:1245:target_handle_connect()) mylustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5508.475733] Lustre: mylustre-OST0000: new connection from mylustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:100 to 0x280000400:129) [ 5509.307694] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5509.492379] Lustre: mylustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 5509.495415] Lustre: Skipped 1 previous similar message [ 5513.393145] Lustre: mylustre-OST0000: new connection from mylustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49699 to 0x280000401:49761) [ 5514.059906] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 5520.826934] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 5523.688828] Lustre: 233685:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 5537.757206] LustreError: 233996:0:(obd_class.h:479:obd_check_dev()) Device 10 not setup [ 5537.767400] LustreError: 233996:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 5537.858929] Lustre: server umount mylustre-MDT0000 complete [ 5539.304360] Lustre: mylustre-MDT0000-lwp-OST0001: Connection to mylustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5539.309404] LustreError: 230724:0:(ldlm_lib.c:1245:target_handle_connect()) mylustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5539.316854] Lustre: Skipped 3 previous similar messages [ 5539.326900] LustreError: 230724:0:(ldlm_lib.c:1245:target_handle_connect()) Skipped 2 previous similar messages [ 5540.425104] LustreError: 230700:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1761289850 with bad export cookie 18262947275277125976 [ 5540.433026] LustreError: 230700:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 5540.433507] LustreError: MGC192.168.201.152@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5540.649404] Lustre: server umount mylustre-MDT0001 complete [ 5553.679247] LustreError: 234398:0:(obd_class.h:479:obd_check_dev()) Device 23 not setup [ 5553.681660] LustreError: 234398:0:(obd_class.h:479:obd_check_dev()) Skipped 15 previous similar messages [ 5553.741943] Lustre: server umount mylustre-OST0000 complete [ 5566.992332] Lustre: server umount mylustre-OST0001 complete [ 5569.511549] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro [ 5569.573119] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5570.371088] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro [ 5570.434517] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5571.160459] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: ,errors=remount-ro [ 5571.214368] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5571.929757] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: ,errors=remount-ro [ 5571.989243] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5582.104492] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing load_modules_local [ 5588.723850] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5589.036677] LustreError: 236009:0:(ldlm_lib.c:1245:target_handle_connect()) tfs-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5589.052042] LustreError: 236009:0:(ldlm_lib.c:1245:target_handle_connect()) Skipped 1 previous similar message [ 5589.117891] Lustre: tfs-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5592.121435] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 5598.790563] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5601.939163] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 5604.032060] Lustre: 237120:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 5608.301711] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5609.580554] Lustre: tfs-OST0000: new connection from tfs-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:131 to 0x280000400:161) [ 5612.101878] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 5616.107216] Lustre: tfs-OST0000: new connection from tfs-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49699 to 0x280000401:49793) [ 5617.548309] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5621.134298] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 5626.802514] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 5629.207097] Lustre: 238966:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 5638.114604] Lustre: tfs-MDT0000-osp-MDT0001: Connection to tfs-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5638.118533] Lustre: tfs-MDT0000: Not available for connect from 0@lo (stopping) [ 5638.122106] Lustre: Skipped 1 previous similar message [ 5643.238182] Lustre: tfs-MDT0000: Not available for connect from 0@lo (stopping) [ 5643.241278] Lustre: Skipped 6 previous similar messages [ 5644.269384] LustreError: 239219:0:(obd_class.h:479:obd_check_dev()) Device 10 not setup [ 5644.273233] LustreError: 239219:0:(obd_class.h:479:obd_check_dev()) Skipped 9 previous similar messages [ 5644.338236] Lustre: server umount tfs-MDT0000 complete [ 5647.098359] LustreError: 235984:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1761289957 with bad export cookie 18262947275277127929 [ 5647.100271] LustreError: MGC192.168.201.152@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5647.114028] LustreError: 235984:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 5647.344651] Lustre: server umount tfs-MDT0001 complete [ 5659.027812] Lustre: server umount tfs-OST0000 complete [ 5671.310972] Lustre: server umount tfs-OST0001 complete [ 5673.580941] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro [ 5673.626646] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5674.400759] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro [ 5674.464651] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5675.179539] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: ,errors=remount-ro [ 5675.281260] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5676.043205] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: ,errors=remount-ro [ 5676.106142] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5684.800572] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing load_modules_local [ 5691.322779] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5691.606083] LustreError: 241231:0:(ldlm_lib.c:1245:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5691.618490] LustreError: 241231:0:(ldlm_lib.c:1245:target_handle_connect()) Skipped 5 previous similar messages [ 5691.653768] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5691.657652] Lustre: Skipped 3 previous similar messages [ 5694.179430] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 5699.381666] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5702.203529] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 5704.051390] Lustre: 242343:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 5708.719663] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5710.067350] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:131 to 0x280000400:193) [ 5712.884204] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 5716.495461] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49699 to 0x280000401:49825) [ 5720.106940] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5724.503289] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 5730.497705] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 5733.186838] Lustre: 244189:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 5743.022985] Lustre: DEBUG MARKER: == conf-sanity test 104a: Make sure user defined options are reflected in mount ========================================================== 03:14:12 (1761290052) [ 5760.994621] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 5761.004649] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5761.013712] Lustre: Skipped 2 previous similar messages [ 5761.018822] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5762.894271] LustreError: 244558:0:(obd_class.h:479:obd_check_dev()) Device 10 not setup [ 5762.901035] LustreError: 244558:0:(obd_class.h:479:obd_check_dev()) Skipped 25 previous similar messages [ 5762.991498] Lustre: server umount lustre-MDT0000 complete [ 5765.712140] LustreError: 244459:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1761290075 with bad export cookie 18262947275277129623 [ 5765.720024] LustreError: MGC192.168.201.152@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5765.722608] LustreError: 244459:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 5766.624924] LustreError: 242137:0:(ldlm_lib.c:1245:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5766.636570] LustreError: 242137:0:(ldlm_lib.c:1245:target_handle_connect()) Skipped 6 previous similar messages [ 5770.660741] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 5770.663788] Lustre: Skipped 7 previous similar messages [ 5772.136865] Lustre: server umount lustre-MDT0001 complete [ 5789.152150] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 5789.217073] Lustre: server umount lustre-OST0000 complete [ 5797.738653] Lustre: server umount lustre-OST0001 complete [ 5801.160767] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_hostid [ 5807.068556] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing load_modules_local [ 5813.908194] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5818.313244] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5822.230772] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5826.824589] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5831.417355] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5831.474918] LDISKFS-fs (dm-0): Mount option "noacl" will be removed by 3.5 [ 5831.474918] Contact linux-ldiskfs@vger.kernel.org if you think we should keep it. [ 5831.474918] [ 5831.490303] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: acl,user_xattr,errors=remount-ro,noacl,no_mbcache,nodelalloc [ 5831.652209] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 5831.691805] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 5831.739951] Lustre: lustre-MDT0000: new disk, initializing [ 5831.840214] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5831.842744] Lustre: Skipped 3 previous similar messages [ 5831.851779] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 5834.581072] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 5841.859116] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5841.904609] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5842.003758] Lustre: 247347:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 5842.044673] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 5842.051433] Lustre: Skipped 1 previous similar message [ 5842.099483] Lustre: lustre-MDT0001: new disk, initializing [ 5842.190213] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 5842.199826] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 5842.260087] Lustre: 248144:0:(osd_internal.h:1361:osd_trans_exec_op()) lustre-MDT0001: opcode 2: before 507 < left 524, rollback = 2 [ 5842.265237] Lustre: 248144:0:(osd_handler.c:2088:osd_trans_dump_creds()) create: 3/12/6, destroy: 0/0/0 [ 5842.269604] Lustre: 248144:0:(osd_handler.c:2095:osd_trans_dump_creds()) attr_set: 3/3/0, xattr_set: 5/524/0 [ 5842.278186] Lustre: 248144:0:(osd_handler.c:2105:osd_trans_dump_creds()) write: 6/142/0, punch: 0/0/0, quota 1/3/0 [ 5842.293727] Lustre: 248144:0:(osd_handler.c:2112:osd_trans_dump_creds()) insert: 7/135/3, delete: 0/0/0 [ 5842.304066] Lustre: 248144:0:(osd_handler.c:2119:osd_trans_dump_creds()) ref_add: 1/1/0, ref_del: 0/0/0 [ 5844.694707] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 5851.958480] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5852.002238] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5852.159175] Lustre: lustre-OST0000: new disk, initializing [ 5852.164886] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 5853.398804] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 5853.406158] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 5853.463504] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 5855.821512] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 5863.324888] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5863.370933] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5863.477975] Lustre: lustre-OST0001: new disk, initializing [ 5863.484040] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 5864.697475] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 5864.706792] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 5864.777881] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 5867.272836] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 5875.770000] Lustre: DEBUG MARKER: == conf-sanity test 104b: Mount uses last flock argument ========================================================== 03:16:25 (1761290185) [ 5884.485783] Lustre: DEBUG MARKER: == conf-sanity test 105: check file creation for ro and rw bind mnt pt ========================================================== 03:16:33 (1761290193) [ 5899.746255] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 5899.750932] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5899.756994] Lustre: Skipped 5 previous similar messages [ 5899.759389] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5903.995839] LustreError: 250770:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 5903.998569] LustreError: 250770:0:(obd_class.h:479:obd_check_dev()) Skipped 25 previous similar messages [ 5904.070360] Lustre: server umount lustre-OST0000 complete [ 5905.891974] LustreError: 249022:0:(ldlm_lib.c:1245:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5905.900108] LustreError: 249022:0:(ldlm_lib.c:1245:target_handle_connect()) Skipped 3 previous similar messages [ 5908.967390] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 5908.971073] LustreError: Skipped 1 previous similar message [ 5912.520541] Lustre: server umount lustre-MDT0000 complete [ 5914.845201] LustreError: 248137:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1761290224 with bad export cookie 18262947275277131177 [ 5914.846105] LustreError: MGC192.168.201.152@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5914.853143] LustreError: 248137:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 5916.131399] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5916.140908] Lustre: Skipped 4 previous similar messages [ 5921.194296] Lustre: server umount lustre-MDT0001 complete [ 5931.619470] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing unload_modules_local [ 5947.452817] Lustre: server umount lustre-OST0001 complete [ 5953.016456] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_hostid [ 5963.629544] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing load_modules_local [ 5970.320867] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5974.692880] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5979.021979] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5982.996263] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5987.673790] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5987.720576] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5987.861771] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 5987.885748] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 5987.940332] Lustre: lustre-MDT0000: new disk, initializing [ 5988.015770] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 5990.829761] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 5998.540213] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5998.603178] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5998.669979] Lustre: 254233:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 5998.961492] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 5999.078041] Lustre: 255034:0:(osd_internal.h:1361:osd_trans_exec_op()) lustre-MDT0001: opcode 2: before 507 < left 524, rollback = 2 [ 5999.085089] Lustre: 255034:0:(osd_internal.h:1361:osd_trans_exec_op()) Skipped 1 previous similar message [ 5999.098783] Lustre: 255034:0:(osd_handler.c:2088:osd_trans_dump_creds()) create: 3/12/6, destroy: 0/0/0 [ 5999.110026] Lustre: 255034:0:(osd_handler.c:2088:osd_trans_dump_creds()) Skipped 1 previous similar message [ 5999.120084] Lustre: 255034:0:(osd_handler.c:2095:osd_trans_dump_creds()) attr_set: 3/3/0, xattr_set: 5/524/0 [ 5999.124635] Lustre: 255034:0:(osd_handler.c:2095:osd_trans_dump_creds()) Skipped 1 previous similar message [ 5999.144392] Lustre: 255034:0:(osd_handler.c:2105:osd_trans_dump_creds()) write: 6/142/0, punch: 0/0/0, quota 1/3/0 [ 5999.157862] Lustre: 255034:0:(osd_handler.c:2105:osd_trans_dump_creds()) Skipped 1 previous similar message [ 5999.175046] Lustre: 255034:0:(osd_handler.c:2112:osd_trans_dump_creds()) insert: 7/135/3, delete: 0/0/0 [ 5999.181971] Lustre: 255034:0:(osd_handler.c:2112:osd_trans_dump_creds()) Skipped 1 previous similar message [ 5999.188740] Lustre: 255034:0:(osd_handler.c:2119:osd_trans_dump_creds()) ref_add: 1/1/0, ref_del: 0/0/0 [ 5999.192832] Lustre: 255034:0:(osd_handler.c:2119:osd_trans_dump_creds()) Skipped 1 previous similar message [ 6001.776973] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 6007.384379] Lustre: DEBUG MARKER: oleg152-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 6010.147917] Lustre: DEBUG MARKER: oleg152-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 6014.874703] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6014.938252] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 6015.219660] Lustre: lustre-OST0000: new disk, initializing [ 6015.223271] Lustre: Skipped 1 previous similar message [ 6015.228702] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 6015.234398] Lustre: Skipped 2 previous similar messages [ 6016.882538] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 6016.887865] Lustre: Skipped 1 previous similar message [ 6016.890576] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 6016.950591] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 6019.528101] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 6025.452942] Lustre: DEBUG MARKER: oleg152-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 6032.357428] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6032.358878] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6032.364589] Lustre: Skipped 1 previous similar message [ 6032.369820] Lustre: Skipped 9 previous similar messages [ 6032.376501] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 6035.574962] LustreError: 256681:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 6035.585133] LustreError: 256681:0:(obd_class.h:479:obd_check_dev()) Skipped 25 previous similar messages [ 6035.707566] Lustre: server umount lustre-OST0000 complete [ 6044.636961] Lustre: server umount lustre-MDT0000 complete [ 6047.507600] LustreError: 255016:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1761290357 with bad export cookie 18262947275277132689 [ 6047.508987] LustreError: MGC192.168.201.152@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6047.518756] LustreError: 255016:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 1 previous similar message [ 6047.866449] Lustre: server umount lustre-MDT0001 complete [ 6055.182405] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing unload_modules_local [ 6057.104183] Key type lgssc unregistered [ 6057.289726] LNet: 257616:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 6057.295563] LNetError: 257616:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 6057.307658] LNet: Removed LNI 192.168.201.152@tcp [ 6057.820213] Key type .llcrypt unregistered [ 6057.821952] Key type ._llcrypt unregistered [ 6075.443264] Key type ._llcrypt registered [ 6075.445197] Key type .llcrypt registered [ 6075.509161] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing load_modules_local [ 6076.069562] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 6076.081808] alg: No test for adler32 (adler32-zlib) [ 6076.988743] Lustre: Lustre: Build Version: 2.16.59_49_g4265256 [ 6077.140192] LNet: Added LNI 192.168.201.152@tcp [8/256/0/180] [ 6078.760627] Key type lgssc registered [ 6079.373924] Lustre: Echo OBD driver; http://www.lustre.org/ [ 6084.457457] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 6084.476321] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6085.797336] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6088.005408] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 6093.327987] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 6093.571589] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 6095.846826] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 6099.039158] Lustre: DEBUG MARKER: oleg152-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 6101.208049] Lustre: DEBUG MARKER: oleg152-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 6105.312640] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 6105.503952] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 6108.809297] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 6110.644262] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:3 to 0x280000401:33) [ 6112.252920] Lustre: DEBUG MARKER: oleg152-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 6112.873816] Lustre: Setting parameter lustre-OST0000.ost.unknown_param=50 in log lustre-OST0000 [ 6115.816755] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 6115.821519] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6115.831580] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6116.093230] LustreError: 261335:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 6116.225571] Lustre: server umount lustre-OST0000 complete [ 6118.967671] LustreError: 261536:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup [ 6118.970693] LustreError: 261536:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 6119.035118] Lustre: server umount lustre-MDT0000 complete [ 6119.393918] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 6119.401828] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6119.413812] LustreError: 259040:0:(ldlm_lib.c:1245:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6119.422893] LustreError: 259040:0:(ldlm_lib.c:1245:target_handle_connect()) Skipped 2 previous similar messages [ 6121.572515] LustreError: 261238:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1761290431 with bad export cookie 8727825225285935627 [ 6121.573924] LustreError: MGC192.168.201.152@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6121.584227] LustreError: 261238:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 6127.690516] LustreError: 261736:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [ 6127.692732] LustreError: 261736:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 6127.816657] Lustre: server umount lustre-MDT0001 complete [ 6133.541751] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing unload_modules_local [ 6135.001169] Key type lgssc unregistered [ 6135.154662] LNet: 262268:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 6135.158785] LNetError: 262268:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 6135.168253] LNet: Removed LNI 192.168.201.152@tcp [ 6135.610391] Key type .llcrypt unregistered [ 6135.612364] Key type ._llcrypt unregistered [ 6146.363700] Key type ._llcrypt registered [ 6146.366133] Key type .llcrypt registered [ 6146.437386] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing load_modules_local [ 6146.961020] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 6147.032264] alg: No test for adler32 (adler32-zlib) [ 6147.988440] Lustre: Lustre: Build Version: 2.16.59_49_g4265256 [ 6148.135995] LNet: Added LNI 192.168.201.152@tcp [8/256/0/180] [ 6149.768352] Key type lgssc registered [ 6150.376335] Lustre: Echo OBD driver; http://www.lustre.org/ [ 6156.185570] Lustre: lustre-OST0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 6156.199767] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 6171.681154] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 6176.800326] LustreError: 263422:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.201.152@tcp: failed processing log, type 4: rc = -110 [ 6203.426734] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 6206.907916] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 6210.305678] Lustre: DEBUG MARKER: oleg152-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 6211.592415] LustreError: 264088:0:(obd_class.h:479:obd_check_dev()) Device 1 not setup [ 6211.700596] Lustre: server umount lustre-OST0000 complete [ 6221.788062] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing unload_modules_local [ 6223.625818] Key type lgssc unregistered [ 6223.860409] LNet: 264922:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 6223.863949] LNetError: 264922:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 6224.940341] LNet: Removed LNI 192.168.201.152@tcp [ 6225.548912] Key type .llcrypt unregistered [ 6225.552558] Key type ._llcrypt unregistered [ 6250.940597] Key type ._llcrypt registered [ 6250.942477] Key type .llcrypt registered [ 6251.041149] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing load_modules_local [ 6251.614995] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 6251.632095] alg: No test for adler32 (adler32-zlib) [ 6252.614299] Lustre: Lustre: Build Version: 2.16.59_49_g4265256 [ 6252.773342] LNet: Added LNI 192.168.201.152@tcp [8/256/0/180] [ 6254.408177] Key type lgssc registered [ 6255.236712] Lustre: Echo OBD driver; http://www.lustre.org/ [ 6263.401135] loop0: detected capacity change from 0 to 204800000 [ 6263.528749] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 6263.536663] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 6263.748209] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 6263.781737] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6264.421442] loop0: detected capacity change from 0 to 204800000 [ 6264.565300] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 6264.575088] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 6264.803728] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 6264.839134] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6265.439081] loop0: detected capacity change from 0 to 204800000 [ 6265.574124] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 6265.579865] blk_update_request: operation not supported error, dev loop0, sector 8224 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 6265.650781] blk_update_request: operation not supported error, dev loop0, sector 58264 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 6265.726429] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6266.409046] loop0: detected capacity change from 0 to 204800000 [ 6266.551874] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 6266.698020] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6267.322440] loop0: detected capacity change from 0 to 204800000 [ 6267.346467] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 6269.894187] loop0: detected capacity change from 0 to 204800000 [ 6269.915663] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 6272.474649] loop0: detected capacity change from 0 to 204800000 [ 6272.501275] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 6275.066383] loop0: detected capacity change from 0 to 204800000 [ 6275.080420] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 6277.480080] loop0: detected capacity change from 0 to 204800000 [ 6277.550045] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6277.593043] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 6277.612529] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6277.648652] Lustre: 268301:0:(osd_scrub.c:1852:osd_index_restore()) lustre-MDT0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 6277.654810] Lustre: 268301:0:(osd_scrub.c:1852:osd_index_restore()) lustre-MDT0000: restore index '0x10000-MDT0000' with [0x200000005:0x1:0x0]: rc = 0 [ 6277.666661] Lustre: 268301:0:(osd_scrub.c:1852:osd_index_restore()) lustre-MDT0000: restore index '0x1010000-MDT0000' with [0x200000005:0x2:0x0]: rc = 0 [ 6277.674228] Lustre: 268301:0:(osd_scrub.c:1852:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000003:0xa:0x0]: rc = 0 [ 6277.681803] Lustre: 268301:0:(osd_scrub.c:1852:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000003:0xb:0x0]: rc = 0 [ 6277.688023] Lustre: 268301:0:(osd_scrub.c:1852:osd_index_restore()) lustre-MDT0000: restore index '0x2010000-MDT0000' with [0x200000005:0x3:0x0]: rc = 0 [ 6277.698728] Lustre: 268301:0:(osd_scrub.c:1852:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000003:0xc:0x0]: rc = 0 [ 6277.710397] Lustre: 268301:0:(osd_scrub.c:1852:osd_index_restore()) lustre-MDT0000: restore index 'nodemap' with [0x200000003:0x3:0x0]: rc = 0 [ 6277.720490] Lustre: 268301:0:(osd_scrub.c:1852:osd_index_restore()) lustre-MDT0000: restore index '0x1020000' with [0x200000006:0x1020000:0x0]: rc = 0 [ 6277.728344] Lustre: 268301:0:(osd_scrub.c:1852:osd_index_restore()) lustre-MDT0000: restore index '0x20000' with [0x200000006:0x20000:0x0]: rc = 0 [ 6277.735283] Lustre: 268301:0:(osd_scrub.c:1852:osd_index_restore()) lustre-MDT0000: restore index '0x2020000' with [0x200000006:0x2020000:0x0]: rc = 0 [ 6277.742266] Lustre: 268301:0:(osd_scrub.c:1852:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000006:0x10000:0x0]: rc = 0 [ 6277.748736] Lustre: 268301:0:(osd_scrub.c:1852:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000006:0x1010000:0x0]: rc = 0 [ 6277.755105] Lustre: 268301:0:(osd_scrub.c:1852:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000006:0x2010000:0x0]: rc = 0 [ 6281.499083] LustreError: 268567:0:(obd_class.h:479:obd_check_dev()) Device 2 not setup [ 6281.598388] Lustre: server umount MGS complete [ 6282.179593] loop0: detected capacity change from 0 to 204800000 [ 6282.251511] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6282.517256] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6282.556323] Lustre: lustre-MDT0000: Aborting client recovery [ 6283.063708] loop1: detected capacity change from 0 to 204800000 [ 6283.135875] LDISKFS-fs (loop1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6283.174155] LDISKFS-fs (loop1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 6283.217520] Lustre: 268739:0:(osd_scrub.c:1852:osd_index_restore()) lustre-MDT0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 6283.223565] Lustre: 268739:0:(osd_scrub.c:1852:osd_index_restore()) lustre-MDT0001: restore index '0x1010000-MDT0001' with [0x200000005:0x2:0x0]: rc = 0 [ 6283.229095] Lustre: 268739:0:(osd_scrub.c:1852:osd_index_restore()) lustre-MDT0001: restore index '0x2010000-MDT0001' with [0x200000005:0x3:0x0]: rc = 0 [ 6283.234786] Lustre: 268739:0:(osd_scrub.c:1852:osd_index_restore()) lustre-MDT0001: restore index '0x1010000' with [0x200000003:0x5:0x0]: rc = 0 [ 6283.241495] Lustre: 268739:0:(osd_scrub.c:1852:osd_index_restore()) lustre-MDT0001: restore index '0x2010000' with [0x200000003:0x6:0x0]: rc = 0 [ 6283.248561] Lustre: 268739:0:(osd_scrub.c:1852:osd_index_restore()) lustre-MDT0001: restore index '0x10000-MDT0001' with [0x200000005:0x1:0x0]: rc = 0 [ 6283.254228] Lustre: 268739:0:(osd_scrub.c:1852:osd_index_restore()) lustre-MDT0001: restore index '0x10000' with [0x200000003:0x4:0x0]: rc = 0 [ 6283.268200] Lustre: Found index 1 for lustre-MDT0001, updating log [ 6283.392116] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 6283.431584] Lustre: lustre-MDT0001: Aborting client recovery [ 6283.458421] Lustre: 268768:0:(osd_internal.h:1361:osd_trans_exec_op()) lustre-MDT0001: opcode 2: before 507 < left 524, rollback = 2 [ 6283.465506] Lustre: 268768:0:(osd_handler.c:2088:osd_trans_dump_creds()) create: 3/12/6, destroy: 0/0/0 [ 6283.469321] Lustre: 268768:0:(osd_handler.c:2095:osd_trans_dump_creds()) attr_set: 3/3/0, xattr_set: 5/524/0 [ 6283.477425] Lustre: 268768:0:(osd_handler.c:2105:osd_trans_dump_creds()) write: 6/89/0, punch: 0/0/0, quota 1/3/0 [ 6283.482425] Lustre: 268768:0:(osd_handler.c:2112:osd_trans_dump_creds()) insert: 7/135/3, delete: 0/0/0 [ 6283.489101] Lustre: 268768:0:(osd_handler.c:2119:osd_trans_dump_creds()) ref_add: 1/1/0, ref_del: 0/0/0 [ 6283.987869] loop2: detected capacity change from 0 to 204800000 [ 6284.097061] LDISKFS-fs (loop2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6284.137522] LDISKFS-fs (loop2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 6284.160625] Lustre: 268824:0:(osd_scrub.c:1852:osd_index_restore()) lustre-OST0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 6284.167503] Lustre: 268824:0:(osd_scrub.c:1852:osd_index_restore()) lustre-OST0000: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 6284.175355] Lustre: 268824:0:(osd_scrub.c:1852:osd_index_restore()) lustre-OST0000: restore index '0x1020000-OST0000' with [0x200000005:0x2:0x0]: rc = 0 [ 6284.182779] Lustre: 268824:0:(osd_scrub.c:1852:osd_index_restore()) lustre-OST0000: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 6284.188950] Lustre: 268824:0:(osd_scrub.c:1852:osd_index_restore()) lustre-OST0000: restore index '0x20000-OST0000' with [0x200000005:0x1:0x0]: rc = 0 [ 6284.194299] Lustre: 268824:0:(osd_scrub.c:1852:osd_index_restore()) lustre-OST0000: restore index '0x2020000-OST0000' with [0x200000005:0x3:0x0]: rc = 0 [ 6284.203457] Lustre: 268824:0:(osd_scrub.c:1852:osd_index_restore()) lustre-OST0000: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 6284.280154] Lustre: Found index 0 for lustre-OST0000, updating log [ 6284.364068] Lustre: lustre-OST0000: Aborting recovery [ 6284.919452] loop3: detected capacity change from 0 to 204800000 [ 6285.016201] LDISKFS-fs (loop3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6285.072728] LDISKFS-fs (loop3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 6285.104250] Lustre: 268912:0:(osd_scrub.c:1852:osd_index_restore()) lustre-OST0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 6285.111825] Lustre: 268912:0:(osd_scrub.c:1852:osd_index_restore()) lustre-OST0001: restore index '0x20000-OST0001' with [0x200000005:0x1:0x0]: rc = 0 [ 6285.116804] Lustre: 268912:0:(osd_scrub.c:1852:osd_index_restore()) lustre-OST0001: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 6285.123802] Lustre: 268912:0:(osd_scrub.c:1852:osd_index_restore()) lustre-OST0001: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 6285.130373] Lustre: 268912:0:(osd_scrub.c:1852:osd_index_restore()) lustre-OST0001: restore index '0x1020000-OST0001' with [0x200000005:0x2:0x0]: rc = 0 [ 6285.137150] Lustre: 268912:0:(osd_scrub.c:1852:osd_index_restore()) lustre-OST0001: restore index '0x2020000-OST0001' with [0x200000005:0x3:0x0]: rc = 0 [ 6285.142938] Lustre: 268912:0:(osd_scrub.c:1852:osd_index_restore()) lustre-OST0001: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 6285.227563] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 6285.233125] Lustre: Skipped 1 previous similar message [ 6285.250239] Lustre: lustre-OST0001: Aborting recovery [ 6290.426301] LustreError: 268681:0:(lod_dev.c:510:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 8, retries 0, failed: rc = -108 [ 6292.457673] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 6292.475689] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 6292.501207] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000bd0 [ 6292.516188] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000bd0 [ 6293.481018] Lustre: lustre-OST0001: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:5 to 0x2c0000400:33) [ 6293.482033] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:5 to 0x280000400:33) [ 6298.197419] LustreError: 268648:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0000: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0000 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0000/identity_upcall, time 1753us: rc = -2 [ 6313.954898] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6313.958734] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6313.962034] Lustre: Skipped 2 previous similar messages [ 6313.965507] Lustre: Skipped 1 previous similar message [ 6313.968940] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 6319.074499] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6319.078189] Lustre: Skipped 4 previous similar messages [ 6324.194947] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6324.195116] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 4. Is it stuck? [ 6324.197405] Lustre: Skipped 3 previous similar messages [ 6324.250615] LustreError: 269262:0:(obd_class.h:479:obd_check_dev()) Device 10 not setup [ 6324.256097] LustreError: 269262:0:(obd_class.h:479:obd_check_dev()) Skipped 1 previous similar message [ 6324.317144] Lustre: server umount lustre-MDT0000 complete [ 6324.788639] LustreError: 268629:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1761290634 with bad export cookie 4320557386997099177 [ 6324.791256] LustreError: MGC192.168.201.152@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6324.798774] LustreError: 268629:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 6329.314234] LustreError: 268645:0:(ldlm_lib.c:1245:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6329.314658] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6329.317243] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 6329.321174] LustreError: 268645:0:(ldlm_lib.c:1245:target_handle_connect()) Skipped 3 previous similar messages [ 6329.326475] Lustre: Skipped 2 previous similar messages [ 6330.950548] LustreError: 269316:0:(obd_class.h:479:obd_check_dev()) Device 18 not setup [ 6330.953720] LustreError: 269316:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 6331.073763] Lustre: server umount lustre-MDT0001 complete [ 6337.658695] LustreError: 269367:0:(obd_class.h:479:obd_check_dev()) Device 23 not setup [ 6337.660916] LustreError: 269367:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 6337.710297] Lustre: server umount lustre-OST0000 complete [ 6344.305132] LustreError: 269418:0:(obd_class.h:479:obd_check_dev()) Device 2 not setup [ 6344.309679] LustreError: 269418:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 6344.419732] Lustre: server umount lustre-OST0001 complete [ 6348.719380] Lustre: DEBUG MARKER: == conf-sanity test 109a: test lctl clear_conf fsname ==== 03:24:18 (1761290658) [ 6361.665678] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_hostid [ 6366.043594] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing load_modules_local [ 6370.702572] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6373.980479] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6376.696814] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6379.838440] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6383.290251] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6383.325415] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6383.443117] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 6383.476305] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 6383.524659] Lustre: lustre-MDT0000: new disk, initializing [ 6383.577169] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6383.592046] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 6385.578400] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 6391.457790] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6391.510081] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 6391.564483] Lustre: 272967:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 6391.611758] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 6391.614796] Lustre: Skipped 1 previous similar message [ 6391.661777] Lustre: lustre-MDT0001: new disk, initializing [ 6391.723355] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 6391.746232] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 6391.754089] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 6391.797052] Lustre: 273761:0:(osd_internal.h:1361:osd_trans_exec_op()) lustre-MDT0001: opcode 2: before 507 < left 524, rollback = 2 [ 6391.803272] Lustre: 273761:0:(osd_internal.h:1361:osd_trans_exec_op()) Skipped 1 previous similar message [ 6391.807405] Lustre: 273761:0:(osd_handler.c:2088:osd_trans_dump_creds()) create: 3/12/6, destroy: 0/0/0 [ 6391.814617] Lustre: 273761:0:(osd_handler.c:2088:osd_trans_dump_creds()) Skipped 1 previous similar message [ 6391.818773] Lustre: 273761:0:(osd_handler.c:2095:osd_trans_dump_creds()) attr_set: 3/3/0, xattr_set: 5/524/0 [ 6391.824941] Lustre: 273761:0:(osd_handler.c:2095:osd_trans_dump_creds()) Skipped 1 previous similar message [ 6391.828310] Lustre: 273761:0:(osd_handler.c:2105:osd_trans_dump_creds()) write: 6/142/0, punch: 0/0/0, quota 1/3/0 [ 6391.833467] Lustre: 273761:0:(osd_handler.c:2105:osd_trans_dump_creds()) Skipped 1 previous similar message [ 6391.839072] Lustre: 273761:0:(osd_handler.c:2112:osd_trans_dump_creds()) insert: 7/135/3, delete: 0/0/0 [ 6391.843303] Lustre: 273761:0:(osd_handler.c:2112:osd_trans_dump_creds()) Skipped 1 previous similar message [ 6391.847642] Lustre: 273761:0:(osd_handler.c:2119:osd_trans_dump_creds()) ref_add: 1/1/0, ref_del: 0/0/0 [ 6391.854692] Lustre: 273761:0:(osd_handler.c:2119:osd_trans_dump_creds()) Skipped 1 previous similar message [ 6393.793695] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 6398.236397] Lustre: DEBUG MARKER: oleg152-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 6399.940622] Lustre: DEBUG MARKER: oleg152-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 6403.624409] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6403.669249] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 6403.803243] Lustre: lustre-OST0000: new disk, initializing [ 6403.807661] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 6403.848651] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 6406.763911] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 6410.255986] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 6410.263112] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 6410.317535] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 6410.679113] Lustre: DEBUG MARKER: oleg152-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 6413.780437] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [ 6420.052762] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [ 6425.128482] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [ 6471.653889] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6471.654514] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 6471.657410] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6471.657420] Lustre: Skipped 1 previous similar message [ 6471.658731] Lustre: Skipped 1 previous similar message [ 6471.662901] LustreError: Skipped 1 previous similar message [ 6477.362246] LustreError: 276482:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 6477.365224] LustreError: 276482:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 6477.448041] Lustre: server umount lustre-OST0000 complete [ 6481.889860] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6481.890797] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6481.897325] Lustre: Skipped 1 previous similar message [ 6481.901813] Lustre: Skipped 4 previous similar messages [ 6485.427080] Lustre: server umount lustre-MDT0000 complete [ 6487.014043] LustreError: 273869:0:(ldlm_lib.c:1245:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6487.021837] LustreError: 273869:0:(ldlm_lib.c:1245:target_handle_connect()) Skipped 1 previous similar message [ 6487.255579] LustreError: 272959:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1761290797 with bad export cookie 4320557386997101921 [ 6487.260177] LustreError: MGC192.168.201.152@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6487.277361] LustreError: 272959:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 6487.467112] Lustre: server umount lustre-MDT0001 complete [ 6491.410965] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6493.077367] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 6495.097821] LustreError: 277825:0:(obd_class.h:479:obd_check_dev()) Device 2 not setup [ 6495.100974] LustreError: 277825:0:(obd_class.h:479:obd_check_dev()) Skipped 17 previous similar messages [ 6495.148713] Lustre: server umount MGS complete [ 6500.233056] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6500.451257] LustreError: 278450:0:(ldlm_lib.c:1245:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6500.463056] LustreError: 278462:0:(qmt_pool.c:1523:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [ 6500.500321] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6502.327362] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 6505.953660] LustreError: 278450:0:(ldlm_lib.c:1245:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6506.106688] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 6507.937802] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 6510.530136] Lustre: DEBUG MARKER: oleg152-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 6512.299551] Lustre: DEBUG MARKER: oleg152-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 6515.667264] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 6518.467392] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 6520.939665] Lustre: DEBUG MARKER: oleg152-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 6547.937407] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 6547.942090] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6547.952229] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6551.017411] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 6553.719355] LustreError: 281111:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 6553.723623] LustreError: 281111:0:(obd_class.h:479:obd_check_dev()) Skipped 1 previous similar message [ 6553.823513] Lustre: server umount lustre-OST0000 complete [ 6556.130655] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6556.136750] Lustre: Skipped 2 previous similar messages [ 6561.900784] Lustre: server umount lustre-MDT0000 complete [ 6563.699397] LustreError: 280848:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1761290873 with bad export cookie 4320557386997103776 [ 6563.702111] LustreError: MGC192.168.201.152@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6563.704838] LustreError: 280848:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 6563.877683] Lustre: server umount lustre-MDT0001 complete [ 6568.604125] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing unload_modules_local [ 6569.885491] Key type lgssc unregistered [ 6570.057711] LNet: 282045:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 6570.062501] LNetError: 282045:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 6570.072388] LNet: Removed LNI 192.168.201.152@tcp [ 6570.536711] Key type .llcrypt unregistered [ 6570.539478] Key type ._llcrypt unregistered [ 6587.404220] Key type ._llcrypt registered [ 6587.405709] Key type .llcrypt registered [ 6587.453721] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_hostid [ 6594.997974] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing load_modules_local [ 6595.401155] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 6595.463495] alg: No test for adler32 (adler32-zlib) [ 6596.366274] Lustre: Lustre: Build Version: 2.16.59_49_g4265256 [ 6596.499896] LNet: Added LNI 192.168.201.152@tcp [8/256/0/180] [ 6598.096410] Key type lgssc registered [ 6598.603737] Lustre: Echo OBD driver; http://www.lustre.org/ [ 6602.498056] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6605.392561] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6608.335095] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6611.803958] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6615.108047] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6615.129440] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 6615.135129] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6616.250413] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 6616.270937] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 6616.317560] Lustre: lustre-MDT0000: new disk, initializing [ 6616.372505] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6616.384044] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 6618.275074] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 6625.177978] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6625.225435] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 6625.307792] Lustre: 285952:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 6625.382375] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 6625.385866] Lustre: Skipped 1 previous similar message [ 6625.468706] Lustre: lustre-MDT0001: new disk, initializing [ 6625.591938] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 6625.631978] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 6625.650802] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 6629.347452] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 6639.569311] Lustre: DEBUG MARKER: oleg152-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 6643.749731] Lustre: DEBUG MARKER: oleg152-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 6652.921541] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6652.982174] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 6653.272724] Lustre: lustre-OST0000: new disk, initializing [ 6653.275881] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 6653.366212] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 6661.551222] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 6663.361760] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 6663.375803] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 6663.516243] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 6672.196308] Lustre: DEBUG MARKER: oleg152-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 6679.657709] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [ 6689.520513] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [ 6696.005672] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [ 6702.920953] Lustre: Modifying parameter lustre-client.llite.max_read_ahead_mb=64 in log lustre-client [ 6740.454891] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 6740.459432] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6740.462804] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6740.464428] LustreError: Skipped 1 previous similar message [ 6740.493757] Lustre: Skipped 1 previous similar message [ 6745.571854] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6745.575560] Lustre: Skipped 2 previous similar messages [ 6746.171377] LustreError: 289420:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 6746.379375] Lustre: server umount lustre-OST0000 complete [ 6750.689947] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6750.694898] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6750.715092] Lustre: Skipped 1 previous similar message [ 6750.730498] Lustre: Skipped 1 previous similar message [ 6755.810442] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6755.818589] Lustre: Skipped 1 previous similar message [ 6756.122545] LustreError: 289623:0:(obd_class.h:479:obd_check_dev()) Device 22 not setup [ 6756.142362] LustreError: 289623:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 6756.264989] Lustre: server umount lustre-MDT0000 complete [ 6760.203223] LustreError: 286732:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1761291070 with bad export cookie 6783806968570627333 [ 6760.210598] LustreError: MGC192.168.201.152@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6760.215060] LustreError: 286732:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 6760.370349] LustreError: 289824:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 6760.374028] LustreError: 289824:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 6760.560671] Lustre: server umount lustre-MDT0001 complete [ 6768.108980] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6771.465903] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 6776.251433] LustreError: 290766:0:(obd_class.h:479:obd_check_dev()) Device 2 not setup [ 6776.254432] LustreError: 290766:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 6776.332486] Lustre: server umount MGS complete [ 6785.750851] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6786.149924] LustreError: 291402:0:(qmt_pool.c:1523:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [ 6786.201607] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6789.491090] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 6791.652931] LustreError: 291389:0:(ldlm_lib.c:1245:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6791.665870] LustreError: 291389:0:(ldlm_lib.c:1245:target_handle_connect()) Skipped 2 previous similar messages [ 6796.771837] LustreError: 291389:0:(ldlm_lib.c:1245:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6797.208859] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 6797.649751] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 6801.353899] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 6807.209253] Lustre: DEBUG MARKER: oleg152-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 6810.854717] Lustre: DEBUG MARKER: oleg152-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 6817.111563] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 6817.401030] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 6822.788144] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 6827.743926] Lustre: DEBUG MARKER: oleg152-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 6857.701813] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 6857.702756] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6857.720612] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6857.738355] Lustre: Skipped 1 previous similar message [ 6857.751952] Lustre: Skipped 1 previous similar message [ 6863.563037] LustreError: 294054:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 6863.574771] LustreError: 294054:0:(obd_class.h:479:obd_check_dev()) Skipped 1 previous similar message [ 6863.736287] Lustre: server umount lustre-OST0000 complete [ 6867.940069] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6867.950707] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6867.954362] Lustre: Skipped 3 previous similar messages [ 6881.248150] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 6881.313602] LustreError: 294255:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup [ 6881.317594] LustreError: 294255:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 6881.384712] Lustre: server umount lustre-MDT0000 complete [ 6883.297834] LustreError: 291386:0:(ldlm_lib.c:1245:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6883.319896] LustreError: 291386:0:(ldlm_lib.c:1245:target_handle_connect()) Skipped 1 previous similar message [ 6884.323604] LustreError: 293789:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1761291194 with bad export cookie 6783806968570629216 [ 6884.332234] LustreError: MGC192.168.201.152@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6884.677103] Lustre: server umount lustre-MDT0001 complete [ 6893.317633] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing unload_modules_local [ 6895.711433] Key type lgssc unregistered [ 6895.968549] LNet: 294989:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 6895.976519] LNetError: 294989:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 6895.997520] LNet: Removed LNI 192.168.201.152@tcp [ 6896.726744] Key type .llcrypt unregistered [ 6896.728426] Key type ._llcrypt unregistered [ 6916.702749] Key type ._llcrypt registered [ 6916.704625] Key type .llcrypt registered [ 6916.823638] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_hostid [ 6928.728288] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing load_modules_local [ 6929.326360] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 6929.427150] alg: No test for adler32 (adler32-zlib) [ 6930.579267] Lustre: Lustre: Build Version: 2.16.59_49_g4265256 [ 6930.865909] LNet: Added LNI 192.168.201.152@tcp [8/256/0/180] [ 6932.592172] Key type lgssc registered [ 6933.765898] Lustre: Echo OBD driver; http://www.lustre.org/ [ 6943.291620] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6949.818814] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6959.215789] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing load_modules_local [ 6967.148807] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6972.747550] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6972.776710] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 6972.788468] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6974.083454] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 6974.122369] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 6974.202165] Lustre: lustre-MDT0000: new disk, initializing [ 6974.315845] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6974.339261] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 6978.184853] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 6987.196817] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6987.279612] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 6987.559126] Lustre: lustre-OST0000: new disk, initializing [ 6987.564055] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 6987.574386] Lustre: Skipped 1 previous similar message [ 6987.657379] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 6992.941498] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 6992.948967] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 6992.993059] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 6993.636913] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 7845.571917] LustreError: 300012:0:(obd_class.h:479:obd_check_dev()) Device 12 not setup [ 7845.807324] Lustre: server umount lustre-MDT0000 complete [ 7849.093063] LustreError: 299827:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1761292159 with bad export cookie 16828974260699942726 [ 7849.101397] LustreError: MGC192.168.201.152@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7859.724952] LustreError: 300217:0:(obd_class.h:479:obd_check_dev()) Device 2 not setup [ 7859.730343] LustreError: 300217:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 7859.881512] Lustre: server umount lustre-OST0000 complete [ 7885.085418] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_hostid [ 7895.338474] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing load_modules_local [ 7904.881282] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 7910.828556] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 7916.198451] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 7922.216413] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 7928.828889] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 7928.902521] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7929.075320] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 7929.102952] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 7929.153372] Lustre: lustre-MDT0000: new disk, initializing [ 7929.233504] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7929.250547] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 7932.801747] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 7942.487782] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 7942.556589] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 7942.630490] Lustre: 303263:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 7942.687686] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 7942.691218] Lustre: Skipped 1 previous similar message [ 7942.755753] Lustre: lustre-MDT0001: new disk, initializing [ 7942.844282] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7942.888088] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 7942.919521] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 7943.014205] Lustre: 304062:0:(osd_internal.h:1361:osd_trans_exec_op()) lustre-MDT0001: opcode 2: before 507 < left 524, rollback = 2 [ 7943.018563] Lustre: 304062:0:(osd_handler.c:2088:osd_trans_dump_creds()) create: 3/12/6, destroy: 0/0/0 [ 7943.041300] Lustre: 304062:0:(osd_handler.c:2095:osd_trans_dump_creds()) attr_set: 3/3/0, xattr_set: 5/524/0 [ 7943.049034] Lustre: 304062:0:(osd_handler.c:2105:osd_trans_dump_creds()) write: 6/142/0, punch: 0/0/0, quota 1/3/0 [ 7943.076341] Lustre: 304062:0:(osd_handler.c:2112:osd_trans_dump_creds()) insert: 7/135/3, delete: 0/0/0 [ 7943.086344] Lustre: 304062:0:(osd_handler.c:2119:osd_trans_dump_creds()) ref_add: 1/1/0, ref_del: 0/0/0 [ 7946.659638] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 7953.382342] Lustre: DEBUG MARKER: oleg152-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 7957.121152] Lustre: DEBUG MARKER: oleg152-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 7963.948356] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 7964.032959] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 7964.238063] Lustre: lustre-OST0000: new disk, initializing [ 7964.244317] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 7964.333787] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7970.047302] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 7971.891940] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 7971.903938] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 7971.998421] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 7977.848708] Lustre: DEBUG MARKER: oleg152-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 7982.832293] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 7983.029578] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 7987.145932] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 7987.329855] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 7992.804938] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7992.810965] Lustre: Skipped 1 previous similar message [ 7992.813609] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7992.817803] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 7995.331309] LustreError: 306210:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 7995.339702] LustreError: 306210:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 7995.545442] Lustre: server umount lustre-OST0000 complete [ 8003.048050] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8003.052957] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8003.062398] Lustre: Skipped 1 previous similar message [ 8003.069359] Lustre: Skipped 1 previous similar message [ 8005.900574] LustreError: 306411:0:(obd_class.h:479:obd_check_dev()) Device 22 not setup [ 8005.905785] LustreError: 306411:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 8005.979745] Lustre: server umount lustre-MDT0000 complete [ 8010.300406] LustreError: 305015:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1761292320 with bad export cookie 16828974260702883587 [ 8010.309643] LustreError: MGC192.168.201.152@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8010.335421] LustreError: 305015:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 8010.488637] LustreError: 306613:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 8010.491239] LustreError: 306613:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 8010.914203] Lustre: server umount lustre-MDT0001 complete [ 8015.508605] Lustre: DEBUG MARKER: SKIP: conf-sanity test_111 skipping SLOW test 111 [ 8017.644341] Lustre: DEBUG MARKER: == conf-sanity test 112a: mount OST with no_create option ========================================================== 03:52:06 (1761292326) [ 8024.465831] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8024.891353] LustreError: 307187:0:(ldlm_lib.c:1245:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8024.909350] LustreError: 307187:0:(ldlm_lib.c:1245:target_handle_connect()) Skipped 3 previous similar messages [ 8025.050094] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8029.185163] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 8030.181637] LustreError: 307187:0:(ldlm_lib.c:1245:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8030.215077] LustreError: 307187:0:(ldlm_lib.c:1245:target_handle_connect()) Skipped 1 previous similar message [ 8035.302858] LustreError: 307187:0:(ldlm_lib.c:1245:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8038.560337] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 8039.117778] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8043.285561] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 8049.135416] Lustre: DEBUG MARKER: oleg152-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 8053.444557] Lustre: DEBUG MARKER: oleg152-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 8060.504418] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 8060.786842] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8067.730941] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 8074.112731] Lustre: DEBUG MARKER: oleg152-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 8081.837444] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 8081.906714] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 8082.072393] Lustre: lustre-OST0001: new disk, initializing [ 8082.079485] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 8082.179848] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 8089.187484] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 8089.327981] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 8089.333247] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 8089.452576] LustreError: 310038:0:(osp_precreate.c:969:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [ 8090.532551] LustreError: 310044:0:(osp_precreate.c:969:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [ 8090.540320] LustreError: 310044:0:(osp_precreate.c:969:osp_precreate_cleanup_orphans()) Skipped 2 previous similar messages [ 8091.560846] LustreError: 310038:0:(osp_precreate.c:969:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [ 8091.571320] LustreError: 310038:0:(osp_precreate.c:969:osp_precreate_cleanup_orphans()) Skipped 1 previous similar message [ 8093.603589] LustreError: 310044:0:(osp_precreate.c:969:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [ 8093.626984] LustreError: 310044:0:(osp_precreate.c:969:osp_precreate_cleanup_orphans()) Skipped 3 previous similar messages [ 8097.704279] LustreError: 310038:0:(osp_precreate.c:969:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [ 8097.725815] LustreError: 310038:0:(osp_precreate.c:969:osp_precreate_cleanup_orphans()) Skipped 6 previous similar messages [ 8099.497199] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 8099.714486] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 8103.445946] Lustre: DEBUG MARKER: oleg152-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff9404860ac000.ost_server_uuid 50 [ 8105.365378] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff9404860ac000.ost_server_uuid in FULL state after 0 sec [ 8106.019097] LustreError: 310044:0:(osp_precreate.c:969:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [ 8106.038267] LustreError: 310044:0:(osp_precreate.c:969:osp_precreate_cleanup_orphans()) Skipped 16 previous similar messages [ 8109.924486] Lustre: DEBUG MARKER: oleg152-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff9404860ac000.ost_server_uuid 50 [ 8112.120208] Lustre: DEBUG MARKER: osc.lustre-OST0001-osc-ffff9404860ac000.ost_server_uuid in FULL state after 0 sec [ 8119.792737] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 8129.514246] LustreError: lustre-OST0001-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 8129.517554] Lustre: lustre-OST0001-osc-MDT0001: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8129.550039] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 8129.553671] Lustre: Skipped 1 previous similar message [ 8135.657073] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 8135.659686] Lustre: Skipped 3 previous similar messages [ 8140.784310] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 8140.789301] Lustre: Skipped 2 previous similar messages [ 8142.304591] Lustre: lustre-OST0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 8142.367460] LustreError: 311000:0:(obd_class.h:479:obd_check_dev()) Device 25 not setup [ 8142.381714] LustreError: 311000:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 8142.471036] Lustre: server umount lustre-OST0001 complete [ 8145.894820] LustreError: 308787:0:(ldlm_lib.c:1245:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8145.933199] LustreError: 308787:0:(ldlm_lib.c:1245:target_handle_connect()) Skipped 1 previous similar message [ 8151.011893] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 8151.020375] LustreError: 308788:0:(ldlm_lib.c:1245:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8151.024555] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8151.024566] Lustre: Skipped 1 previous similar message [ 8151.028450] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8151.028460] Lustre: Skipped 1 previous similar message [ 8151.032630] LustreError: Skipped 1 previous similar message [ 8151.076457] LustreError: 308788:0:(ldlm_lib.c:1245:target_handle_connect()) Skipped 2 previous similar messages [ 8154.931255] Lustre: server umount lustre-OST0000 complete [ 8161.250990] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8161.272359] Lustre: Skipped 2 previous similar messages [ 8165.028560] LustreError: 311406:0:(obd_class.h:479:obd_check_dev()) Device 28 not setup [ 8165.043492] LustreError: 311406:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 8165.189933] Lustre: server umount lustre-MDT0000 complete [ 8166.374300] LustreError: 307186:0:(ldlm_lib.c:1245:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8166.395738] LustreError: 307186:0:(ldlm_lib.c:1245:target_handle_connect()) Skipped 1 previous similar message [ 8169.204782] LustreError: 307168:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1761292479 with bad export cookie 16828974260702884511 [ 8169.214624] LustreError: MGC192.168.201.152@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8169.218214] LustreError: 307168:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 8169.761655] Lustre: server umount lustre-MDT0001 complete [ 8181.429960] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing unload_modules_local [ 8184.366584] Key type lgssc unregistered [ 8184.696784] LNet: 312141:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8184.712219] LNetError: 312141:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8185.767140] LNet: Removed LNI 192.168.201.152@tcp [ 8186.661603] Key type .llcrypt unregistered [ 8186.669655] Key type ._llcrypt unregistered [ 8214.194191] Key type ._llcrypt registered [ 8214.195990] Key type .llcrypt registered [ 8214.307678] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing load_modules_local [ 8215.019730] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8215.095072] alg: No test for adler32 (adler32-zlib) [ 8216.199556] Lustre: Lustre: Build Version: 2.16.59_49_g4265256 [ 8216.455141] LNet: Added LNI 192.168.201.152@tcp [8/256/0/180] [ 8218.130444] Key type lgssc registered [ 8219.237741] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8227.077128] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 8227.113343] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8228.712557] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8232.553032] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 8238.051536] LustreError: 313547:0:(ldlm_lib.c:1245:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8238.057864] LustreError: 313547:0:(ldlm_lib.c:1245:target_handle_connect()) Skipped 2 previous similar messages [ 8238.084609] Lustre: DEBUG MARKER: oleg152-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 8243.193142] LustreError: 313547:0:(ldlm_lib.c:1245:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8244.381649] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 8244.696025] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8248.166090] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 8259.640039] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 8260.294478] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8268.802262] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 8269.491447] LustreError: 315103:0:(ldlm_lib.c:1245:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8273.595780] LustreError: 315474:0:(ldlm_lib.c:1245:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8273.616813] LustreError: 315474:0:(ldlm_lib.c:1245:target_handle_connect()) Skipped 1 previous similar message [ 8273.651831] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:65) [ 8277.406836] Lustre: DEBUG MARKER: oleg152-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 8279.013488] LustreError: 315474:0:(ldlm_lib.c:1245:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8279.024277] LustreError: 315474:0:(ldlm_lib.c:1245:target_handle_connect()) Skipped 1 previous similar message [ 8284.715506] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 8285.046229] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 8290.308936] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:33) [ 8290.733950] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 8296.133178] Lustre: DEBUG MARKER: oleg152-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid [ 8301.937648] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing wait_import_state (FULL|IDLE) os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [ 8302.154584] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 8413.154578] LustreError: lustre-OST0001-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 8413.155549] Lustre: lustre-OST0001-osc-MDT0001: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8413.160458] LustreError: Skipped 1 previous similar message [ 8413.170030] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 8413.180889] Lustre: Skipped 1 previous similar message [ 8415.304093] LustreError: 316978:0:(obd_class.h:479:obd_check_dev()) Device 27 not setup [ 8415.398725] Lustre: server umount lustre-OST0001 complete [ 8418.276742] LustreError: 315474:0:(ldlm_lib.c:1245:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8418.299833] LustreError: 315474:0:(ldlm_lib.c:1245:target_handle_connect()) Skipped 3 previous similar messages [ 8422.371811] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 8422.379096] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8422.386799] Lustre: Skipped 1 previous similar message [ 8422.394504] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8423.402692] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 8423.403023] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8423.403228] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8423.418716] Lustre: Skipped 1 previous similar message [ 8426.810200] LustreError: 317180:0:(obd_class.h:479:obd_check_dev()) Device 23 not setup [ 8426.812608] LustreError: 317180:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 8426.982734] Lustre: server umount lustre-OST0000 complete [ 8433.633688] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8433.671345] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8433.688203] Lustre: Skipped 1 previous similar message [ 8436.714837] LustreError: 317381:0:(obd_class.h:479:obd_check_dev()) Device 10 not setup [ 8436.721508] LustreError: 317381:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 8436.831841] Lustre: server umount lustre-MDT0000 complete [ 8438.754743] LustreError: 316295:0:(ldlm_lib.c:1245:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8438.771587] LustreError: 316295:0:(ldlm_lib.c:1245:target_handle_connect()) Skipped 3 previous similar messages [ 8440.176431] LustreError: 313530:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1761292750 with bad export cookie 2674421463113401067 [ 8440.185740] LustreError: MGC192.168.201.152@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8440.364061] LustreError: 317583:0:(obd_class.h:479:obd_check_dev()) Device 18 not setup [ 8440.380165] LustreError: 317583:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 8440.614235] Lustre: server umount lustre-MDT0001 complete [ 8450.534050] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing unload_modules_local [ 8453.434692] Key type lgssc unregistered [ 8453.692844] LNet: 318114:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8453.697205] LNetError: 318114:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8453.709208] LNet: Removed LNI 192.168.201.152@tcp [ 8454.477264] Key type .llcrypt unregistered [ 8454.479581] Key type ._llcrypt unregistered [ 8503.573495] Key type ._llcrypt registered [ 8503.584390] Key type .llcrypt registered [ 8503.725712] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing load_modules_local [ 8504.786122] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8505.061771] alg: No test for adler32 (adler32-zlib) [ 8507.248622] Lustre: Lustre: Build Version: 2.16.59_49_g4265256 [ 8507.616921] LNet: Added LNI 192.168.201.152@tcp [8/256/0/180] [ 8509.384515] Key type lgssc registered [ 8510.850812] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8520.448877] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 8520.473382] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8522.053991] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8525.745965] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 8532.455147] LustreError: 320391:0:(ldlm_lib.c:1245:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8532.466962] LustreError: 320391:0:(ldlm_lib.c:1245:target_handle_connect()) Skipped 2 previous similar messages [ 8535.378724] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 8535.964668] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8539.938825] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 8543.410933] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 8549.884196] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 8550.183910] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8556.131210] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 8560.426480] LustreError: 321853:0:(ldlm_lib.c:1245:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8565.236500] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 8565.519204] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 8567.625108] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:97) [ 8567.628526] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:65) [ 8571.366120] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 8581.470950] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 8590.784868] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 8607.713253] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8607.722234] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8607.743868] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8608.742471] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8608.744768] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8608.753324] Lustre: Skipped 1 previous similar message [ 8608.757942] Lustre: Skipped 1 previous similar message [ 8613.190449] LustreError: 323815:0:(obd_class.h:479:obd_check_dev()) Device 10 not setup [ 8613.320681] Lustre: server umount lustre-MDT0000 complete [ 8613.861868] LustreError: 321678:0:(ldlm_lib.c:1245:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8613.868744] LustreError: 321678:0:(ldlm_lib.c:1245:target_handle_connect()) Skipped 3 previous similar messages [ 8616.962551] LustreError: 323358:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1761292927 with bad export cookie 5040807982405295742 [ 8616.967148] LustreError: MGC192.168.201.152@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8617.010393] LustreError: 323358:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 8617.194823] LustreError: 324016:0:(obd_class.h:479:obd_check_dev()) Device 18 not setup [ 8617.272297] LustreError: 324016:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 8617.665827] Lustre: server umount lustre-MDT0001 complete [ 8631.690143] LustreError: 324217:0:(obd_class.h:479:obd_check_dev()) Device 23 not setup [ 8631.698244] LustreError: 324217:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 8631.779628] Lustre: server umount lustre-OST0000 complete [ 8645.258364] LustreError: 324419:0:(obd_class.h:479:obd_check_dev()) Device 2 not setup [ 8645.260547] LustreError: 324419:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 8645.443191] Lustre: server umount lustre-OST0001 complete [ 8659.636757] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing load_modules_local [ 8669.034216] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8669.419613] LustreError: 325597:0:(ldlm_lib.c:1245:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8669.517574] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8672.992242] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 8674.789818] LustreError: 325597:0:(ldlm_lib.c:1245:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8680.059111] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 8680.449414] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8683.603841] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 8686.077331] Lustre: 326708:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 8691.519934] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 8694.914039] LustreError: 327063:0:(ldlm_lib.c:1245:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8694.920040] LustreError: 327063:0:(ldlm_lib.c:1245:target_handle_connect()) Skipped 1 previous similar message [ 8694.931673] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:129) [ 8696.500123] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 8703.723330] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 8703.951377] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 8703.965969] Lustre: Skipped 1 previous similar message [ 8705.052330] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:97) [ 8708.804869] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 8716.287263] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 8720.063079] Lustre: 328556:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 8730.600596] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8730.601647] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8730.614033] Lustre: Skipped 2 previous similar messages [ 8730.616631] Lustre: Skipped 2 previous similar messages [ 8734.081651] LustreError: 328662:0:(obd_class.h:479:obd_check_dev()) Device 10 not setup [ 8734.101137] LustreError: 328662:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 8734.230987] Lustre: server umount lustre-MDT0000 complete [ 8735.716726] LustreError: 325597:0:(ldlm_lib.c:1245:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8735.729483] LustreError: 325597:0:(ldlm_lib.c:1245:target_handle_connect()) Skipped 4 previous similar messages [ 8737.401282] LustreError: 325577:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1761293047 with bad export cookie 5040807982405297443 [ 8737.405799] LustreError: MGC192.168.201.152@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8737.414543] LustreError: 325577:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 8737.853895] Lustre: server umount lustre-MDT0001 complete [ 8752.190760] LustreError: 329066:0:(obd_class.h:479:obd_check_dev()) Device 23 not setup [ 8752.195650] LustreError: 329066:0:(obd_class.h:479:obd_check_dev()) Skipped 15 previous similar messages [ 8752.269556] Lustre: server umount lustre-OST0000 complete [ 8765.211990] Lustre: server umount lustre-OST0001 complete [ 8768.867782] Lustre: DEBUG MARKER: SKIP: conf-sanity test_114 skipping SLOW test 114 [ 8770.494842] Lustre: DEBUG MARKER: SKIP: conf-sanity test_115 skipping excluded test 115 [ 8772.046820] Lustre: DEBUG MARKER: == conf-sanity test 116: big size MDT support ============ 04:04:41 (1761293081) [ 8791.001956] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing load_modules_local [ 8797.482739] loop0: detected capacity change from 0 to 1099511627776 [ 8797.737270] SGI XFS with ACLs, security attributes, quota, no debug enabled [ 8797.744931] *** VALIDATE xfs *** [ 8797.753779] XFS (loop0): Mounting V5 Filesystem [ 8797.777432] XFS (loop0): Ending clean mount [ 8798.916577] loop1: detected capacity change from 0 to 18691697672192 [ 8850.692129] LDISKFS-fs (loop1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 8866.220402] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_hostid [ 8873.194224] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing load_modules_local [ 8880.404198] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 8885.757121] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 8891.661702] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 8897.673966] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 8904.307615] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 8904.372755] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8904.565770] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 8904.595337] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 8904.649408] Lustre: lustre-MDT0000: new disk, initializing [ 8904.725558] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8904.749964] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 8908.506067] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 8918.213895] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 8918.280780] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 8918.359158] Lustre: 333649:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 8918.397577] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 8918.400145] Lustre: Skipped 1 previous similar message [ 8918.467627] Lustre: lustre-MDT0001: new disk, initializing [ 8918.581476] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 8918.588036] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 8918.640820] Lustre: 334446:0:(osd_internal.h:1361:osd_trans_exec_op()) lustre-MDT0001: opcode 2: before 507 < left 524, rollback = 2 [ 8918.649878] Lustre: 334446:0:(osd_handler.c:2088:osd_trans_dump_creds()) create: 3/12/6, destroy: 0/0/0 [ 8918.655977] Lustre: 334446:0:(osd_handler.c:2095:osd_trans_dump_creds()) attr_set: 3/3/0, xattr_set: 5/524/0 [ 8918.661864] Lustre: 334446:0:(osd_handler.c:2105:osd_trans_dump_creds()) write: 6/142/0, punch: 0/0/0, quota 1/3/0 [ 8918.667617] Lustre: 334446:0:(osd_handler.c:2112:osd_trans_dump_creds()) insert: 7/135/3, delete: 0/0/0 [ 8918.680995] Lustre: 334446:0:(osd_handler.c:2119:osd_trans_dump_creds()) ref_add: 1/1/0, ref_del: 0/0/0 [ 8921.633565] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 8928.126465] Lustre: DEBUG MARKER: oleg152-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 8931.316738] Lustre: DEBUG MARKER: oleg152-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 8937.979525] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 8938.050677] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 8938.353340] Lustre: lustre-OST0000: new disk, initializing [ 8938.367971] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 8939.925210] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 8939.935772] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 8940.005593] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 8943.514357] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 8950.221984] Lustre: DEBUG MARKER: oleg152-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 8954.191712] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 8954.406487] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 8958.467404] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 8958.666468] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 8965.607417] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 8965.608417] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8965.610217] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8965.610224] Lustre: Skipped 3 previous similar messages [ 8965.621442] LustreError: Skipped 1 previous similar message [ 8965.626734] Lustre: Skipped 3 previous similar messages [ 8966.698129] LustreError: 336591:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 8966.706041] LustreError: 336591:0:(obd_class.h:479:obd_check_dev()) Skipped 9 previous similar messages [ 8966.901517] Lustre: server umount lustre-OST0000 complete [ 8970.722586] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8970.739691] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8970.742699] Lustre: Skipped 1 previous similar message [ 8976.750409] Lustre: server umount lustre-MDT0000 complete [ 8980.044697] LustreError: 333642:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1761293290 with bad export cookie 5040807982405298990 [ 8980.047582] LustreError: MGC192.168.201.152@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8980.128979] LustreError: 333642:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 8980.597828] Lustre: server umount lustre-MDT0001 complete [ 8989.048475] XFS (loop0): Unmounting Filesystem [ 8994.839544] Lustre: DEBUG MARKER: == conf-sanity test 117: lctl get_param return errors properly ========================================================== 04:08:23 (1761293303) [ 9001.192971] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9001.551514] LustreError: 337814:0:(ldlm_lib.c:1245:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9001.629388] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9001.632272] Lustre: Skipped 2 previous similar messages [ 9005.152664] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 9013.475760] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 9017.814101] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 9023.250388] Lustre: DEBUG MARKER: oleg152-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 9027.417655] Lustre: DEBUG MARKER: oleg152-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 9034.145207] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 9039.896692] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 9045.490060] Lustre: DEBUG MARKER: oleg152-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 9059.303546] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 9059.307264] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9059.312523] Lustre: Skipped 1 previous similar message [ 9059.337467] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9059.344281] Lustre: Skipped 2 previous similar messages [ 9062.370523] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 9064.466208] LustreError: 340161:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 9064.471308] LustreError: 340161:0:(obd_class.h:479:obd_check_dev()) Skipped 17 previous similar messages [ 9064.713579] Lustre: server umount lustre-OST0000 complete [ 9070.054516] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 9074.604263] Lustre: server umount lustre-MDT0000 complete [ 9077.731294] LustreError: 338543:0:(ldlm_lib.c:1245:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9077.753416] LustreError: 338543:0:(ldlm_lib.c:1245:target_handle_connect()) Skipped 3 previous similar messages [ 9078.560161] LustreError: 337795:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1761293388 with bad export cookie 5040807982405299914 [ 9078.572388] LustreError: MGC192.168.201.152@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9078.581038] LustreError: 337795:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 9079.028118] Lustre: server umount lustre-MDT0001 complete [ 9088.545566] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing unload_modules_local [ 9091.131472] Key type lgssc unregistered [ 9091.431274] LNet: 341096:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 9091.439266] LNetError: 341096:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 9091.465417] LNet: Removed LNI 192.168.201.152@tcp [ 9092.212299] Key type .llcrypt unregistered [ 9092.214810] Key type ._llcrypt unregistered [ 9115.582576] Key type ._llcrypt registered [ 9115.584078] Key type .llcrypt registered [ 9115.706866] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing load_modules_local [ 9117.058155] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 9117.078890] alg: No test for adler32 (adler32-zlib) [ 9118.149268] Lustre: Lustre: Build Version: 2.16.59_49_g4265256 [ 9118.321971] LNet: Added LNI 192.168.201.152@tcp [8/256/0/180] [ 9119.976195] Key type lgssc registered [ 9121.057842] Lustre: Echo OBD driver; http://www.lustre.org/ [ 9128.390756] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 9128.410166] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9129.899346] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9133.893377] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 9139.175286] LustreError: 342646:0:(ldlm_lib.c:1245:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9139.205833] LustreError: 342646:0:(ldlm_lib.c:1245:target_handle_connect()) Skipped 2 previous similar messages [ 9142.544406] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 9142.811548] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9146.605267] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 9151.889122] Lustre: DEBUG MARKER: oleg152-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 9155.472548] Lustre: DEBUG MARKER: oleg152-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 9161.554053] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 9161.815030] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9166.894498] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 9172.181485] Lustre: DEBUG MARKER: oleg152-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 9180.273957] Lustre: lustre-MDT0001: Not available for connect from 192.168.201.52@tcp (stopping) [ 9183.716440] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [ 9183.739845] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9183.767050] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 9185.372842] Lustre: lustre-MDT0001: Not available for connect from 192.168.201.52@tcp (stopping) [ 9185.382922] Lustre: Skipped 2 previous similar messages [ 9186.153391] LustreError: 344942:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [ 9186.282096] Lustre: server umount lustre-MDT0001 complete [ 9189.116087] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 9189.347272] LustreError: 342646:0:(ldlm_lib.c:1245:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9189.384746] LustreError: 342646:0:(ldlm_lib.c:1245:target_handle_connect()) Skipped 1 previous similar message [ 9190.491606] LustreError: 342642:0:(ldlm_lib.c:1245:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.201.52@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9194.468748] LustreError: 342646:0:(ldlm_lib.c:1245:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9194.493151] LustreError: 342646:0:(ldlm_lib.c:1245:target_handle_connect()) Skipped 1 previous similar message [ 9197.224456] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 9197.316192] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [ 9197.322377] Lustre: Found index 1 for lustre-MDT0001, updating log [ 9197.367061] Lustre: 342632:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 9197.670324] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9201.384724] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 9203.185954] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 9207.284386] LustreError: 345924:0:(obd_mount.c:188:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [ 9207.301928] LustreError: 345924:0:(tgt_mount.c:669:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [ 9207.325671] LustreError: 342637:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [ 9207.429466] LustreError: 345927:0:(obd_config.c:1997:class_config_llog_handler()) MGC192.168.201.152@tcp: cfg command failed: rc = -17 [ 9207.435110] Lustre: cmd=cf001 0:lustre-MDT0001-osp-MDT0000 1:osp 2:lustre-MDT0000-mdtlov_UUID [ 9207.435110] [ 9507.943357] Lustre: lustre-MDT0001: Not available for connect from 192.168.201.52@tcp (stopping) [ 9510.372497] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [ 9510.375775] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9510.388856] Lustre: Skipped 1 previous similar message [ 9511.466883] LustreError: 354561:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [ 9511.481322] LustreError: 354561:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 9511.554231] Lustre: server umount lustre-MDT0001 complete [ 9513.046371] LustreError: 344264:0:(ldlm_lib.c:1245:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.201.52@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9513.065401] LustreError: 344264:0:(ldlm_lib.c:1245:target_handle_connect()) Skipped 1 previous similar message [ 9513.672058] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 9518.449669] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 9518.526475] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [ 9518.533506] Lustre: Found index 1 for lustre-MDT0001, updating log [ 9518.568954] Lustre: 342634:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 9518.793593] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9521.303837] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 9524.199013] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 9526.769922] LustreError: 355608:0:(obd_mount.c:188:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [ 9526.780712] LustreError: 355608:0:(tgt_mount.c:669:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [ 9526.791803] LustreError: 342637:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [ 9526.796510] LustreError: 342637:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [ 9526.852205] LustreError: 355620:0:(obd_config.c:689:class_setup()) Device 8 already setup (type osp) [ 9526.860543] LustreError: 355620:0:(obd_config.c:1997:class_config_llog_handler()) MGC192.168.201.152@tcp: cfg command failed: rc = -17 [ 9526.865994] Lustre: cmd=cf003 0:lustre-MDT0001-osp-MDT0000 1:lustre-MDT0001_UUID 2:192.168.201.152@tcp [ 9526.865994] [ 9825.382883] Lustre: lustre-MDT0001: Not available for connect from 192.168.201.52@tcp (stopping) [ 9825.386415] Lustre: Skipped 2 previous similar messages [ 9826.280962] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9826.283079] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [ 9826.288034] Lustre: Skipped 2 previous similar messages [ 9830.286979] LustreError: 365381:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [ 9830.289870] LustreError: 365381:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 9830.337083] Lustre: server umount lustre-MDT0001 complete [ 9830.483111] LustreError: 346078:0:(ldlm_lib.c:1245:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.201.52@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9830.496301] LustreError: 346078:0:(ldlm_lib.c:1245:target_handle_connect()) Skipped 3 previous similar messages [ 9831.741084] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 9835.458830] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 9835.512696] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [ 9835.515615] Lustre: Found index 1 for lustre-MDT0001, updating log [ 9835.537430] Lustre: 342635:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 9835.602097] Lustre: lustre-MDT0001: Not available for connect from 192.168.201.52@tcp (not set up) [ 9835.606814] Lustre: Skipped 2 previous similar messages [ 9835.700672] Lustre: lustre-MDT0001: Imperative Recovery enabled, recovery window shrunk from 60-180 down to 60-180 [ 9837.546707] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [ 9841.132074] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 9841.145153] LustreError: 366408:0:(obd_mount.c:188:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [ 9841.149519] LustreError: 366408:0:(tgt_mount.c:669:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [ 9841.154976] LustreError: 342637:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [ 9841.158618] LustreError: 342637:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [ 9841.198287] LustreError: 366411:0:(lod_lov.c:207:lod_add_device()) lustre-MDT0001-osp-MDT0000: cannot connect to next dev lustre-MDT0001_UUID (-114) [ 9841.207870] LustreError: 366411:0:(obd_config.c:1997:class_config_llog_handler()) MGC192.168.201.152@tcp: cfg command failed: rc = -114 [ 9841.213554] Lustre: cmd=cf014 0:lustre-MDT0000-mdtlov 1:lustre-MDT0001_UUID 2:1 3:1 [ 9841.213554] [10153.441058] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [10153.444273] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10153.451339] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10158.149922] LustreError: 376816:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup [10158.151990] LustreError: 376816:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [10158.188082] Lustre: server umount lustre-MDT0000 complete [10159.837106] LustreError: 345509:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1761294469 with bad export cookie 6794683915349891612 [10159.840079] LustreError: MGC192.168.201.152@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10159.843240] LustreError: 345509:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [10161.633078] LustreError: 342641:0:(ldlm_lib.c:1245:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10161.633380] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10161.640948] LustreError: 342641:0:(ldlm_lib.c:1245:target_handle_connect()) Skipped 2 previous similar messages [10161.647243] Lustre: Skipped 2 previous similar messages [10165.986514] LustreError: 377018:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [10165.992191] LustreError: 377018:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [10166.138677] Lustre: server umount lustre-MDT0001 complete [10168.249262] Lustre: server umount lustre-OST0000 complete [10172.606968] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_hostid [10176.738339] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing load_modules_local [10180.782461] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [10183.570414] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [10186.320327] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [10189.442462] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [10192.812336] Lustre: DEBUG MARKER: == conf-sanity test 120: cross-target rename should not create bad symlinks ========================================================== 04:28:22 (1761294502) [10195.958516] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [10195.987836] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10196.088921] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [10196.106516] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [10196.148323] Lustre: lustre-MDT0000: new disk, initializing [10196.184587] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10196.193580] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [10197.880365] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [10203.371689] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [10203.410542] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [10203.457322] Lustre: 379750:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [10203.490516] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [10203.493242] Lustre: Skipped 1 previous similar message [10203.535168] Lustre: lustre-MDT0001: new disk, initializing [10203.587595] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10203.610935] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [10203.619208] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [10205.370447] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [10209.060719] Lustre: DEBUG MARKER: oleg152-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [10210.608411] Lustre: DEBUG MARKER: oleg152-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [10213.676640] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [10213.718918] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [10213.846731] Lustre: lustre-OST0000: new disk, initializing [10213.850172] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [10213.898645] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10216.470419] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_default_debug -1 all [10220.413697] Lustre: DEBUG MARKER: oleg152-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [10222.081958] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [10222.086815] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [10222.138886] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [10227.328987] Lustre: 379758:0:(osd_internal.h:1361:osd_trans_exec_op()) lustre-MDT0001: opcode 2: before 507 < left 525, rollback = 2 [10227.334254] Lustre: 379758:0:(osd_handler.c:2088:osd_trans_dump_creds()) create: 3/12/6, destroy: 0/0/0 [10227.337601] Lustre: 379758:0:(osd_handler.c:2095:osd_trans_dump_creds()) attr_set: 3/3/0, xattr_set: 6/525/0 [10227.341469] Lustre: 379758:0:(osd_handler.c:2105:osd_trans_dump_creds()) write: 7/153/0, punch: 0/0/0, quota 1/3/0 [10227.345586] Lustre: 379758:0:(osd_handler.c:2112:osd_trans_dump_creds()) insert: 7/135/3, delete: 0/0/0 [10227.348958] Lustre: 379758:0:(osd_handler.c:2119:osd_trans_dump_creds()) ref_add: 1/1/0, ref_del: 0/0/0 [10229.217057] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [10229.220182] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10229.227114] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10229.233854] Lustre: Skipped 5 previous similar messages [10235.081499] LustreError: 382200:0:(obd_class.h:479:obd_check_dev()) Device 22 not setup [10235.084056] LustreError: 382200:0:(obd_class.h:479:obd_check_dev()) Skipped 11 previous similar messages [10235.125615] Lustre: server umount lustre-MDT0000 complete [10236.899487] LustreError: 379742:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1761294547 with bad export cookie 6794683915349893579 [10236.903408] LustreError: MGC192.168.201.152@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10236.906773] LustreError: 379742:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [10237.088153] Lustre: server umount lustre-MDT0001 complete [10248.082270] Lustre: server umount lustre-OST0000 complete [10260.577406] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing set_hostid [10265.122953] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing load_modules_local [10269.620951] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [10272.762189] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [10275.812721] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [10278.878856] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [10279.642763] Lustre: DEBUG MARKER: == conf-sanity test complete, duration 10055 sec ========= 04:29:49 (1761294589) [10280.417632] Lustre: DEBUG MARKER: === conf-sanity: start cleanup 04:29:50 (1761294590) === [10282.162261] Lustre: DEBUG MARKER: === conf-sanity: finish cleanup 04:29:51 (1761294591) === [10303.967410] Lustre: DEBUG MARKER: oleg152-server.virtnet: executing unload_modules_local [10305.411282] Key type lgssc unregistered [10305.576680] LNet: 387706:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [10305.580586] LNetError: 387706:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [10305.599705] LNet: Removed LNI 192.168.201.152@tcp [10306.050102] Key type .llcrypt unregistered [10306.051739] Key type ._llcrypt unregistered