[ 0.000000] Linux version 4.18.0rh8.10-debug (green@maintenance) (gcc version 8.5.0 20210514 (Red Hat 8.5.0-26) (GCC)) #2 SMP Mon Jul 14 01:24:22 EDT 2025 [ 0.000000] Command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers' [ 0.000000] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256 [ 0.000000] x86/fpu: Enabled xstate features 0x7, context size is 832 bytes, using 'standard' format. [ 0.000000] signal: max sigframe size: 1776 [ 0.000000] BIOS-provided physical RAM map: [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x00000000bffcdfff] usable [ 0.000000] BIOS-e820: [mem 0x00000000bffce000-0x00000000bfffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x0000000146dfffff] usable [ 0.000000] NX (Execute Disable) protection: active [ 0.000000] SMBIOS 2.8 present. [ 0.000000] DMI: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 0.000000] Hypervisor detected: KVM [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00 [ 0.000000] kvm-clock: using sched offset of 560558362 cycles [ 0.000000] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns [ 0.000000] tsc: Detected 2399.998 MHz processor [ 0.000000] last_pfn = 0x146e00 max_arch_pfn = 0x400000000 [ 0.000000] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT [ 0.000000] last_pfn = 0xbffce max_arch_pfn = 0x400000000 [ 0.000000] found SMP MP-table at [mem 0x000f54b0-0x000f54bf] [ 0.000000] RAMDISK: [mem 0xbcc54000-0xbffbffff] [ 0.000000] ACPI: Early table checksum verification disabled [ 0.000000] ACPI: RSDP 0x00000000000F52D0 000014 (v00 BOCHS ) [ 0.000000] ACPI: RSDT 0x00000000BFFE247C 000034 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACP 0x00000000BFFE2318 000074 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: DSDT 0x00000000BFFE0040 0022D8 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACS 0x00000000BFFE0000 000040 [ 0.000000] ACPI: APIC 0x00000000BFFE238C 000090 (v03 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: HPET 0x00000000BFFE241C 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: WAET 0x00000000BFFE2454 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: Reserving FACP table memory at [mem 0xbffe2318-0xbffe238b] [ 0.000000] ACPI: Reserving DSDT table memory at [mem 0xbffe0040-0xbffe2317] [ 0.000000] ACPI: Reserving FACS table memory at [mem 0xbffe0000-0xbffe003f] [ 0.000000] ACPI: Reserving APIC table memory at [mem 0xbffe238c-0xbffe241b] [ 0.000000] ACPI: Reserving HPET table memory at [mem 0xbffe241c-0xbffe2453] [ 0.000000] ACPI: Reserving WAET table memory at [mem 0xbffe2454-0xbffe247b] [ 0.000000] No NUMA configuration found [ 0.000000] Faking a node at [mem 0x0000000000000000-0x0000000146dfffff] [ 0.000000] NODE_DATA(0) allocated [mem 0x1465a3000-0x1465cdfff] [ 0.000000] Reserving 256MB of memory at 2752MB for crashkernel (System RAM: 4205MB) [ 0.000000] Zone ranges: [ 0.000000] DMA [mem 0x0000000000001000-0x0000000000ffffff] [ 0.000000] DMA32 [mem 0x0000000001000000-0x00000000ffffffff] [ 0.000000] Normal [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Device empty [ 0.000000] Movable zone start for each node [ 0.000000] Early memory node ranges [ 0.000000] node 0: [mem 0x0000000000001000-0x000000000009efff] [ 0.000000] node 0: [mem 0x0000000000100000-0x00000000bffcdfff] [ 0.000000] node 0: [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Zeroed struct page in unavailable ranges: 4756 pages [ 0.000000] Initmem setup node 0 [mem 0x0000000000001000-0x0000000146dfffff] [ 0.000000] ACPI: PM-Timer IO Port: 0x608 [ 0.000000] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1]) [ 0.000000] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23 [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level) [ 0.000000] Using ACPI (MADT) for SMP configuration information [ 0.000000] ACPI: HPET id: 0x8086a201 base: 0xfed00000 [ 0.000000] TSC deadline timer available [ 0.000000] smpboot: Allowing 4 CPUs, 0 hotplug CPUs [ 0.000000] kvm-guest: KVM setup pv remote TLB flush [ 0.000000] kvm-guest: setup PV sched yield [ 0.000000] PM: Registered nosave memory: [mem 0x00000000-0x00000fff] [ 0.000000] PM: Registered nosave memory: [mem 0x0009f000-0x0009ffff] [ 0.000000] PM: Registered nosave memory: [mem 0x000a0000-0x000effff] [ 0.000000] PM: Registered nosave memory: [mem 0x000f0000-0x000fffff] [ 0.000000] PM: Registered nosave memory: [mem 0xbffce000-0xbfffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xc0000000-0xfeffbfff] [ 0.000000] PM: Registered nosave memory: [mem 0xfeffc000-0xfeffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xff000000-0xfffbffff] [ 0.000000] PM: Registered nosave memory: [mem 0xfffc0000-0xffffffff] [ 0.000000] [mem 0xc0000000-0xfeffbfff] available for PCI devices [ 0.000000] Booting paravirtualized kernel on KVM [ 0.000000] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns [ 0.000000] setup_percpu: NR_CPUS:8192 nr_cpumask_bits:4 nr_cpu_ids:4 nr_node_ids:1 [ 0.000000] percpu: Embedded 63 pages/cpu s221184 r8192 d28672 u524288 [ 0.000000] kvm-guest: PV spinlocks enabled [ 0.000000] PV qspinlock hash table entries: 256 (order: 0, 4096 bytes, linear) [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 1059606 [ 0.000000] Policy zone: Normal [ 0.000000] Kernel command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] Specific versions of hardware are certified with Red Hat Enterprise Linux 8. Please see the list of hardware certified with Red Hat Enterprise Linux 8 at https://catalog.redhat.com. [ 0.000000] audit: disabled (until reboot) [ 0.000000] software IO TLB: area num 4. [ 0.000000] Memory: 2829652K/4306352K available (18435K kernel code, 11221K rwdata, 7248K rodata, 2908K init, 18040K bss, 524580K reserved, 0K cma-reserved) [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1 [ 0.000000] kmemleak: Kernel memory leak detector disabled [ 0.000000] ftrace: allocating 41240 entries in 162 pages [ 0.000000] ftrace: allocated 162 pages with 3 groups [ 0.000000] rcu: Hierarchical RCU implementation. [ 0.000000] rcu: RCU event tracing is enabled. [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=8192 to nr_cpu_ids=4. [ 0.000000] rcu: RCU callback double-/use-after-free debug enabled. [ 0.000000] Rude variant of Tasks RCU enabled. [ 0.000000] Tracing variant of Tasks RCU enabled. [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies. [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4 [ 0.000000] NR_IRQS: 524544, nr_irqs: 456, preallocated irqs: 16 [ 0.000000] random: get_random_bytes called from start_kernel+0x622/0x9a8 with crng_init=0 [ 0.001000] Console: colour *CGA 80x25 [ 0.001000] printk: console [ttyS1] enabled [ 0.001000] ACPI: Core revision 20220331 [ 0.001000] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns [ 0.001010] APIC: Switch to symmetric I/O mode setup [ 0.002000] x2apic enabled [ 0.002000] Switched APIC routing to physical x2apic. [ 0.002000] kvm-guest: setup PV IPIs [ 0.003000] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1 [ 0.003000] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 0.003029] Calibrating delay loop (skipped) preset value.. 4799.99 BogoMIPS (lpj=2399998) [ 0.004014] pid_max: default: 32768 minimum: 301 [ 0.005203] LSM: Security Framework initializing [ 0.006051] Yama: becoming mindful. [ 0.007031] SELinux: Initializing. [ 0.009032] *** VALIDATE selinux *** [ 0.018216] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, vmalloc) [ 0.024638] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, vmalloc) [ 0.025144] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.026090] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.027089] *** VALIDATE tmpfs *** [ 0.029434] *** VALIDATE proc *** [ 0.031025] *** VALIDATE cgroup *** [ 0.032006] *** VALIDATE cgroup2 *** [ 0.034145] x86/cpu: User Mode Instruction Prevention (UMIP) activated [ 0.035132] Last level iTLB entries: 4KB 0, 2MB 0, 4MB 0 [ 0.036008] Last level dTLB entries: 4KB 0, 2MB 0, 4MB 0, 1GB 0 [ 0.037033] Spectre V2 : User space: Vulnerable [ 0.038007] Speculative Store Bypass: Vulnerable [ 0.045000] debug: unmapping init [mem 0xffffffffaaa59000-0xffffffffaaa60fff] [ 0.048453] smpboot: CPU0: Intel(R) Xeon(R) CPU E5-2695 v2 @ 2.40GHz (family: 0x6, model: 0x3e, stepping: 0x4) [ 0.051999] Performance Events: IvyBridge events, full-width counters, Intel PMU driver. [ 0.053039] ... version: 2 [ 0.054007] ... bit width: 48 [ 0.055008] ... generic registers: 4 [ 0.056008] ... value mask: 0000ffffffffffff [ 0.057009] ... max period: 00007fffffffffff [ 0.058008] ... fixed-purpose events: 3 [ 0.059008] ... event mask: 000000070000000f [ 0.060305] rcu: Hierarchical SRCU implementation. [ 0.062547] smp: Bringing up secondary CPUs ... [ 0.063592] x86: Booting SMP configuration: [ 0.064017] .... node #0, CPUs: #1 #2 #3 [ 0.078173] smp: Brought up 1 node, 4 CPUs [ 0.080010] smpboot: Max logical packages: 1 [ 0.081028] smpboot: Total of 4 processors activated (19199.98 BogoMIPS) [ 0.119935] node 0 deferred pages initialised in 36ms [ 0.122326] devtmpfs: initialized [ 0.123267] x86/mm: Memory block size: 128MB [ 0.126229] gcov: version magic: 0x41383552 [ 0.128322] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns [ 0.129083] futex hash table entries: 1024 (order: 4, 65536 bytes, vmalloc) [ 0.130315] pinctrl core: initialized pinctrl subsystem [ 0.131231] [ 0.131911] ************************************************************* [ 0.132008] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.133008] ** ** [ 0.134017] ** IOMMU DebugFS SUPPORT HAS BEEN ENABLED IN THIS KERNEL ** [ 0.135010] ** ** [ 0.136009] ** This means that this kernel is built to expose internal ** [ 0.137011] ** IOMMU data structures, which may compromise security on ** [ 0.138010] ** your system. ** [ 0.139015] ** ** [ 0.140023] ** If you see this message and you are not debugging the ** [ 0.141011] ** kernel, report this immediately to your vendor! ** [ 0.142077] ** ** [ 0.143015] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.144010] ************************************************************* [ 0.145893] NET: Registered protocol family 16 [ 0.146471] DMA: preallocated 512 KiB GFP_KERNEL pool for atomic allocations [ 0.147076] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations [ 0.148054] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations [ 0.149779] cpuidle: using governor menu [ 0.153166] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5 [ 0.156604] PCI: Using configuration type 1 for base access [ 0.159157] core: PMU erratum BJ122, BV98, HSD29 worked around, HT is on [ 0.176213] HugeTLB registered 1.00 GiB page size, pre-allocated 0 pages [ 0.177013] HugeTLB registered 2.00 MiB page size, pre-allocated 0 pages [ 0.179046] cryptd: max_cpu_qlen set to 1000 [ 0.181380] ACPI: Added _OSI(Module Device) [ 0.182011] ACPI: Added _OSI(Processor Device) [ 0.183010] ACPI: Added _OSI(3.0 _SCP Extensions) [ 0.184009] ACPI: Added _OSI(Processor Aggregator Device) [ 0.190098] ACPI: 1 ACPI AML tables successfully acquired and loaded [ 0.194205] ACPI: Interpreter enabled [ 0.195049] ACPI: PM: (supports S0 S3 S4 S5) [ 0.196008] ACPI: Using IOAPIC for interrupt routing [ 0.197132] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug [ 0.198408] ACPI: Enabled 2 GPEs in block 00 to 0F [ 0.211622] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff]) [ 0.212118] acpi PNP0A03:00: _OSC: OS supports [ASPM ClockPM Segments MSI HPX-Type3] [ 0.213019] acpi PNP0A03:00: _OSC: not requesting OS control; OS requires [ExtendedConfig ASPM ClockPM MSI] [ 0.214078] acpi PNP0A03:00: fail to add MMCONFIG information, can't access extended PCI configuration space under this bridge. [ 0.216887] acpiphp: Slot [2] registered [ 0.217144] acpiphp: Slot [5] registered [ 0.218107] acpiphp: Slot [6] registered [ 0.219219] acpiphp: Slot [7] registered [ 0.220159] acpiphp: Slot [8] registered [ 0.221110] acpiphp: Slot [9] registered [ 0.222137] acpiphp: Slot [10] registered [ 0.223178] acpiphp: Slot [3] registered [ 0.224189] acpiphp: Slot [4] registered [ 0.225178] acpiphp: Slot [11] registered [ 0.226144] acpiphp: Slot [12] registered [ 0.227148] acpiphp: Slot [13] registered [ 0.228101] acpiphp: Slot [14] registered [ 0.229085] acpiphp: Slot [15] registered [ 0.230117] acpiphp: Slot [16] registered [ 0.231204] acpiphp: Slot [17] registered [ 0.232204] acpiphp: Slot [18] registered [ 0.233157] acpiphp: Slot [19] registered [ 0.234110] acpiphp: Slot [20] registered [ 0.235111] acpiphp: Slot [21] registered [ 0.236102] acpiphp: Slot [22] registered [ 0.237090] acpiphp: Slot [23] registered [ 0.238086] acpiphp: Slot [24] registered [ 0.239320] acpiphp: Slot [25] registered [ 0.241101] acpiphp: Slot [26] registered [ 0.243086] acpiphp: Slot [27] registered [ 0.244293] acpiphp: Slot [28] registered [ 0.247129] acpiphp: Slot [29] registered [ 0.249160] acpiphp: Slot [30] registered [ 0.251160] acpiphp: Slot [31] registered [ 0.253084] PCI host bridge to bus 0000:00 [ 0.255024] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window] [ 0.260036] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window] [ 0.263043] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window] [ 0.267023] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window] [ 0.272052] pci_bus 0000:00: root bus resource [mem 0xe0000000000-0xe007fffffff window] [ 0.276059] pci_bus 0000:00: root bus resource [bus 00-ff] [ 0.279546] pci 0000:00:00.0: [8086:1237] type 00 class 0x060000 [ 0.284249] pci 0000:00:01.0: [8086:7000] type 00 class 0x060100 [ 0.288525] pci 0000:00:01.1: [8086:7010] type 00 class 0x010180 [ 0.298841] pci 0000:00:01.1: reg 0x20: [io 0xc320-0xc32f] [ 0.305056] pci 0000:00:01.1: legacy IDE quirk: reg 0x10: [io 0x01f0-0x01f7] [ 0.308020] pci 0000:00:01.1: legacy IDE quirk: reg 0x14: [io 0x03f6] [ 0.311022] pci 0000:00:01.1: legacy IDE quirk: reg 0x18: [io 0x0170-0x0177] [ 0.313020] pci 0000:00:01.1: legacy IDE quirk: reg 0x1c: [io 0x0376] [ 0.316750] pci 0000:00:01.3: [8086:7113] type 00 class 0x068000 [ 0.321013] pci 0000:00:01.3: quirk: [io 0x0600-0x063f] claimed by PIIX4 ACPI [ 0.324088] pci 0000:00:01.3: quirk: [io 0x0700-0x070f] claimed by PIIX4 SMB [ 0.328382] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 [ 0.337019] pci 0000:00:02.0: reg 0x10: [io 0xc300-0xc31f] [ 0.358015] pci 0000:00:02.0: reg 0x20: [mem 0xe0000000000-0xe0000003fff 64bit pref] [ 0.367027] pci 0000:00:02.0: reg 0x30: [mem 0xfeb80000-0xfebbffff pref] [ 0.379068] pci 0000:00:05.0: [1af4:1001] type 00 class 0x010000 [ 0.392026] pci 0000:00:05.0: reg 0x10: [io 0xc000-0xc07f] [ 0.403037] pci 0000:00:05.0: reg 0x14: [mem 0xfebc0000-0xfebc0fff] [ 0.443000] pci 0000:00:05.0: reg 0x20: [mem 0xe0000004000-0xe0000007fff 64bit pref] [ 0.444000] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 [ 0.457000] pci 0000:00:06.0: reg 0x10: [io 0xc080-0xc0ff] [ 0.467016] pci 0000:00:06.0: reg 0x14: [mem 0xfebc1000-0xfebc1fff] [ 0.494015] pci 0000:00:06.0: reg 0x20: [mem 0xe0000008000-0xe000000bfff 64bit pref] [ 0.504089] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 [ 0.514016] pci 0000:00:07.0: reg 0x10: [io 0xc100-0xc17f] [ 0.526012] pci 0000:00:07.0: reg 0x14: [mem 0xfebc2000-0xfebc2fff] [ 0.558017] pci 0000:00:07.0: reg 0x20: [mem 0xe000000c000-0xe000000ffff 64bit pref] [ 0.571112] pci 0000:00:08.0: [1af4:1001] type 00 class 0x010000 [ 0.582013] pci 0000:00:08.0: reg 0x10: [io 0xc180-0xc1ff] [ 0.591020] pci 0000:00:08.0: reg 0x14: [mem 0xfebc3000-0xfebc3fff] [ 0.610012] pci 0000:00:08.0: reg 0x20: [mem 0xe0000010000-0xe0000013fff 64bit pref] [ 0.621206] pci 0000:00:09.0: [1af4:1001] type 00 class 0x010000 [ 0.631021] pci 0000:00:09.0: reg 0x10: [io 0xc200-0xc27f] [ 0.642018] pci 0000:00:09.0: reg 0x14: [mem 0xfebc4000-0xfebc4fff] [ 0.658018] pci 0000:00:09.0: reg 0x20: [mem 0xe0000014000-0xe0000017fff 64bit pref] [ 0.671189] pci 0000:00:0a.0: [1af4:1001] type 00 class 0x010000 [ 0.682177] pci 0000:00:0a.0: reg 0x10: [io 0xc280-0xc2ff] [ 0.693021] pci 0000:00:0a.0: reg 0x14: [mem 0xfebc5000-0xfebc5fff] [ 0.714114] pci 0000:00:0a.0: reg 0x20: [mem 0xe0000018000-0xe000001bfff 64bit pref] [ 0.724166] ACPI: PCI: Interrupt link LNKA configured for IRQ 10 [ 0.726379] ACPI: PCI: Interrupt link LNKB configured for IRQ 10 [ 0.728623] ACPI: PCI: Interrupt link LNKC configured for IRQ 11 [ 0.732400] ACPI: PCI: Interrupt link LNKD configured for IRQ 11 [ 0.735200] ACPI: PCI: Interrupt link LNKS configured for IRQ 9 [ 0.740173] iommu: Default domain type: Passthrough [ 0.741000] SCSI subsystem initialized [ 0.741483] ACPI: bus type USB registered [ 0.744087] usbcore: registered new interface driver usbfs [ 0.748086] usbcore: registered new interface driver hub [ 0.752252] usbcore: registered new device driver usb [ 0.755386] pps_core: LinuxPPS API ver. 1 registered [ 0.758010] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti [ 0.763059] PTP clock support registered [ 0.767003] EDAC MC: Ver: 3.0.0 [ 0.771125] PCI: Using ACPI for IRQ routing [ 0.775741] NetLabel: Initializing [ 0.779028] NetLabel: domain hash size = 128 [ 0.785073] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO [ 0.797234] NetLabel: unlabeled traffic allowed by default [ 0.804925] vgaarb: loaded [ 0.808982] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0 [ 0.811026] hpet0: 3 comparators, 64-bit 100.000000 MHz counter [ 0.819251] clocksource: Switched to clocksource kvm-clock [ 1.021525] VFS: Disk quotas dquot_6.6.0 [ 1.024033] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) [ 1.027047] *** VALIDATE ramfs *** [ 1.028408] *** VALIDATE hugetlbfs *** [ 1.030428] pnp: PnP ACPI init [ 1.033343] pnp: PnP ACPI: found 6 devices [ 1.053060] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns [ 1.057374] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window] [ 1.059930] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window] [ 1.062822] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window] [ 1.066207] pci_bus 0000:00: resource 7 [mem 0xc0000000-0xfebfffff window] [ 1.068937] pci_bus 0000:00: resource 8 [mem 0xe0000000000-0xe007fffffff window] [ 1.072362] NET: Registered protocol family 2 [ 1.075658] IP idents hash table entries: 131072 (order: 8, 1048576 bytes, vmalloc) [ 1.081544] tcp_listen_portaddr_hash hash table entries: 4096 (order: 5, 163840 bytes, vmalloc) [ 1.086206] TCP established hash table entries: 65536 (order: 7, 524288 bytes, vmalloc) [ 1.092370] TCP bind hash table entries: 65536 (order: 9, 2097152 bytes, vmalloc) [ 1.096315] TCP: Hash tables configured (established 65536 bind 65536) [ 1.099642] MPTCP token hash table entries: 8192 (order: 6, 393216 bytes, vmalloc) [ 1.103499] UDP hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 1.107144] UDP-Lite hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 1.110564] NET: Registered protocol family 1 [ 1.113710] RPC: Registered named UNIX socket transport module. [ 1.121164] RPC: Registered udp transport module. [ 1.124194] RPC: Registered tcp transport module. [ 1.126228] RPC: Registered tcp NFSv4.1 backchannel transport module. [ 1.128514] NET: Registered protocol family 44 [ 1.130598] pci 0000:00:00.0: Limiting direct PCI/PCI transfers [ 1.133148] pci 0000:00:01.0: PIIX3: Enabling Passive Release [ 1.135181] pci 0000:00:01.0: Activating ISA DMA hang workarounds [ 1.138404] PCI: CLS 0 bytes, default 64 [ 1.140962] Unpacking initramfs... [ 2.899431] debug: unmapping init [mem 0xffff9ffd3cc54000-0xffff9ffd3ffbffff] [ 2.903701] PCI-DMA: Using software bounce buffering for IO (SWIOTLB) [ 2.905773] software IO TLB: mapped [mem 0x00000000a8000000-0x00000000ac000000] (64MB) [ 2.908778] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 3.712111] Initialise system trusted keyrings [ 3.713767] Key type blacklist registered [ 3.716161] workingset: timestamp_bits=36 max_order=20 bucket_order=0 [ 3.726582] zbud: loaded [ 3.729715] *** VALIDATE nfs *** [ 3.730982] *** VALIDATE nfs4 *** [ 3.735497] pstore: using deflate compression [ 3.740066] Platform Keyring initialized [ 3.868962] NET: Registered protocol family 38 [ 3.871298] Key type asymmetric registered [ 3.873473] Asymmetric key parser 'x509' registered [ 3.875510] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 247) [ 3.879498] io scheduler mq-deadline registered [ 3.881737] io scheduler kyber registered [ 3.883915] io scheduler bfq registered [ 3.886282] atomic64_test: passed for x86-64 platform with CX8 and with SSE [ 3.889940] shpchp: Standard Hot Plug PCI Controller Driver version: 0.4 [ 3.893509] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0 [ 3.897087] ACPI: Power Button [PWRF] [ 3.906086] ACPI: \_SB_.LNKB: Enabled at IRQ 10 [ 3.918351] ACPI: \_SB_.LNKA: Enabled at IRQ 11 [ 3.941569] ACPI: \_SB_.LNKC: Enabled at IRQ 11 [ 3.952937] ACPI: \_SB_.LNKD: Enabled at IRQ 10 [ 3.975743] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled [ 4.011081] 00:03: ttyS1 at I/O 0x2f8 (irq = 3, base_baud = 115200) is a 16550A [ 4.043947] 00:04: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A [ 4.050705] Non-volatile memory driver v1.3 [ 4.052553] Linux agpgart interface v0.103 [ 4.098729] virtio_blk virtio1: [vda] 146544 512-byte logical blocks (75.0 MB/71.6 MiB) [ 4.104180] vda: detected capacity change from 0 to 75030528 [ 4.125818] virtio_blk virtio2: [vdb] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) [ 4.131078] vdb: detected capacity change from 0 to 1073741824 [ 4.153617] virtio_blk virtio3: [vdc] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 4.156986] vdc: detected capacity change from 0 to 2621440000 [ 4.223770] virtio_blk virtio4: [vdd] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 4.228926] vdd: detected capacity change from 0 to 2621440000 [ 4.253316] virtio_blk virtio5: [vde] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 4.256655] vde: detected capacity change from 0 to 4294967296 [ 4.275537] virtio_blk virtio6: [vdf] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 4.279981] vdf: detected capacity change from 0 to 4294967296 [ 4.295462] libphy: Fixed MDIO Bus: probed [ 4.305674] usbcore: registered new interface driver usbserial_generic [ 4.308144] usbserial: USB Serial support registered for generic [ 4.310643] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12 [ 4.316146] serio: i8042 KBD port at 0x60,0x64 irq 1 [ 4.318963] serio: i8042 AUX port at 0x60,0x64 irq 12 [ 4.322715] mousedev: PS/2 mouse device common for all mice [ 4.327690] rtc_cmos 00:05: RTC can wake from S4 [ 4.332739] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input1 [ 4.338520] rtc_cmos 00:05: registered as rtc0 [ 4.338579] rtc_cmos 00:05: alarms up to one day, y3k, 242 bytes nvram, hpet irqs [ 4.338615] intel_pstate: CPU model not supported [ 4.342528] hid: raw HID events driver (C) Jiri Kosina [ 4.352175] usbcore: registered new interface driver usbhid [ 4.354135] usbhid: USB HID core driver [ 4.355667] drop_monitor: Initializing network drop monitor service [ 4.359604] Initializing XFRM netlink socket [ 4.361269] NET: Registered protocol family 10 [ 4.368166] Segment Routing with IPv6 [ 4.370083] NET: Registered protocol family 17 [ 4.372122] mpls_gso: MPLS GSO support [ 4.377204] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input4 [ 4.385368] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input3 [ 4.392747] RAS: Correctable Errors collector initialized. [ 4.395020] AVX version of gcm_enc/dec engaged. [ 4.396889] AES CTR mode by8 optimization enabled [ 4.492927] sched_clock: Marking stable (4492908820, 0)->(5821429119, -1328520299) [ 4.496648] registered taskstats version 1 [ 4.498351] Loading compiled-in X.509 certificates [ 4.500990] zswap: loaded using pool lzo/zbud [ 4.532729] Key type big_key registered [ 4.546533] Key type encrypted registered [ 4.548279] ima: No TPM chip found, activating TPM-bypass! [ 4.549576] ima: Allocated hash algorithm: sha1 [ 4.550574] ima: No architecture policies found [ 4.552173] evm: Initialising EVM extended attributes: [ 4.554114] evm: security.selinux [ 4.555587] evm: security.ima [ 4.557677] evm: security.capability [ 4.559335] evm: HMAC attrs: 0x1 [ 4.565423] rtc_cmos 00:05: setting system clock to 2026-08-22 04:27:39 UTC (1787372859) [ 4.575629] debug: unmapping init [mem 0xffffffffaba03000-0xffffffffabbfffff] [ 4.581196] debug: unmapping init [mem 0xffffffffaa782000-0xffffffffaaa58fff] [ 4.591269] Write protecting the kernel read-only data: 28672k [ 4.596349] debug: unmapping init [mem 0xffffffffa8e03000-0xffffffffa8ffffff] [ 4.601286] debug: unmapping init [mem 0xffffffffa9714000-0xffffffffa97fffff] [ 4.655421] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 4.669450] systemd[1]: Detected virtualization kvm. [ 4.671614] systemd[1]: Detected architecture x86-64. [ 4.674191] systemd[1]: Running in initial RAM disk. Welcome to Rocky Linux 8.10 (Green Obsidian) dracut-049-233.git20240115.el8 (Initramfs)! [ 4.710651] systemd[1]: No hostname configured. [ 4.712621] systemd[1]: Set hostname to . [ 4.714792] random: systemd: uninitialized urandom read (16 bytes read) [ 4.718328] systemd[1]: Initializing machine ID from random generator. [ 4.914041] random: ln: uninitialized urandom read (6 bytes read) [ 5.132326] random: systemd: uninitialized urandom read (16 bytes read) [ 5.134771] systemd[1]: Reached target Initrd Root Device. [ OK ] Reached target Initrd Root Device. [ 5.142984] systemd[1]: Reached target Slices. [ OK ] Reached target Slices. [ 5.151373] systemd[1]: Listening on Journal Socket (/dev/log). [ OK ] Listening on Journal Socket (/dev/log). [ OK ] Listening on udev Control Socket. [ OK ] Listening on Journal Socket. Starting Create list of required st…ce nodes for the current kernel... Starting Journal Service... Starting Apply Kernel Variables... [ OK ] Started Memstrack Anylazing Service. [ OK ] Reached target Timers. [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Reached target Local Encrypted Volumes. Starting Setup Virtual Console... [ OK ] Reached target Paths. [ OK ] Reached target Swap. [ OK ] Reached target Local File Systems. Starting Create Volatile Files and Directories... [ OK ] Listening on udev Kernel Socket. [ OK ] Reached target Sockets. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Started Apply Kernel Variables. [ OK ] Started Setup Virtual Console. [ OK ] Started Create Volatile Files and Directories. Starting dracut cmdline hook... Starting Create Static Device Nodes in /dev... [ OK ] Started Create Static Device Nodes in /dev. [ OK ] Started Journal Service. [ OK ] Started dracut cmdline hook. Starting dracut pre-udev hook... [ 6.282309] device-mapper: uevent: version 1.0.3 [ 6.284532] device-mapper: ioctl: 4.46.0-ioctl (2022-02-22) initialised: dm-devel@redhat.com [ OK ] Started dracut pre-udev hook. Starting udev Kernel Device Manager... [ OK ] Started udev Kernel Device Manager. Starting dracut pre-trigger hook... [ OK ] Started dracut pre-trigger hook. Starting udev Coldplug all Devices... Mounting Kernel Configuration File System... [ OK ] Mounted Kernel Configuration File System. [ OK ] Started udev Coldplug all Devices. Starting dracut initqueue hook... [ OK ] Reached target System Initialization. [ OK ] Reached target Basic System. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. [ 7.819898] virtio_net virtio0 ens2: renamed from eth0 [ 7.862720] random: fast init done [ 8.050093] scsi host0: ata_piix [ 8.072277] scsi host1: ata_piix [ 8.075751] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc320 irq 14 [ 8.083483] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc328 irq 15 [ 13.436348] random: crng init done [ 13.437502] random: 7 urandom warning(s) missed due to ratelimiting [ 16.038983] dracut-initqueue[581]: RTNETLINK answers: File exists [ 16.644315] hrtimer: interrupt took 12461952 ns Starting nbd nbd0... [ OK ] Started nbd nbd0. [ OK ] Started dracut initqueue hook. Mounting /sysroot... [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. [ 17.858422] EXT4-fs (nbd0): mounted filesystem with ordered data mode. Opts: (null) [ OK ] Mounted /sysroot. [ OK ] Reached target Initrd Root File System. Starting Reload Configuration from the Real Root... [ OK ] Started Reload Configuration from the Real Root. [ OK ] Reached target Initrd File Systems. [ OK ] Reached target Initrd Default Target. Starting dracut pre-pivot and cleanup hook... [ OK ] Started dracut pre-pivot and cleanup hook. Starting Cleaning Up and Shutting Down Daemons... [ OK ] Stopped dracut pre-pivot and cleanup hook. [ OK ] Stopped target Timers. [ OK ] Stopped target Initrd Default Target. [ OK ] Stopped target Initrd Root Device. Stopping Hardware RNG Entropy Gatherer Daemon... [ OK ] Stopped target Remote File Systems. [ OK ] Stopped target Remote File Systems (Pre). [ OK ] Stopped dracut initqueue hook. [ OK ] Stopped Hardware RNG Entropy Gatherer Daemon. [ OK ] Stopped target Basic System. [ OK ] Stopped target Sockets. [ OK ] Stopped target Slices. [ OK ] Stopped target Paths. [ OK ] Stopped target System Initialization. [ OK ] Stopped target Local Encrypted Volumes. [ OK ] Stopped Dispatch Password Requests to Console Directory Watch. [ OK ] Stopped Apply Kernel Variables. [ OK ] Stopped udev Coldplug all Devices. [ OK ] Stopped dracut pre-trigger hook. Stopping udev Kernel Device Manager... [ OK ] Stopped target Swap. [ OK ] Stopped Create Volatile Files and Directories. [ OK ] Stopped target Local File Systems. [ OK ] Stopped udev Kernel Device Manager. [ OK ] Started Cleaning Up and Shutting Down Daemons. [ OK ] Stopped dracut pre-udev hook. [ OK ] Stopped dracut cmdline hook. [ OK ] Stopped Create Static Device Nodes in /dev. [ OK ] Stopped Create list of required sta…vice nodes for the current kernel. [ OK ] Closed udev Kernel Socket. [ OK ] Closed udev Control Socket. Starting Cleanup udevd DB... [ OK ] Started Cleanup udevd DB. [ OK ] Reached target Switch Root. Starting Switch Root... [ 20.793390] printk: systemd: 26 output lines suppressed due to ratelimiting [ 21.546213] SELinux: Disabled at runtime. [ 21.740360] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 21.748429] systemd[1]: Detected virtualization kvm. [ 21.750302] systemd[1]: Detected architecture x86-64. Welcome to Rocky Linux 8.10 (Green Obsidian)! [ 23.189308] systemd[1]: initrd-switch-root.service: Succeeded. [ 23.206889] systemd[1]: Stopped Switch Root. [ OK ] Stopped Switch Root. [ 23.228482] systemd[1]: systemd-journald.service: Service has no hold-off time (RestartSec=0), scheduling restart. [ 23.250886] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. [ 23.277304] systemd[1]: Stopped Journal Service. [ OK ] Stopped Journal Service. [ 23.366520] systemd[1]: Starting Journal Service... Starting Journal Service... [ 23.425485] systemd[1]: Created slice system-getty.slice. [ OK ] Created slice system-getty.slice. [ OK ] Listening on udev Control Socket. Starting Remount Root and Kernel File Systems... Mounting Huge Pages File System... [ OK ] Reached target rpc_pipefs.target. [ OK ] Listening on udev Kernel Socket. [ OK ] Started Forward Password Requests to Wall Directory Watch. [ OK ] Stopped target Switch Root. Starting udev Coldplug all Devices... [ OK ] Listening on Process Core Dump Socket. [ OK ] Stopped target Initrd File Systems. [ OK ] Listening on initctl Compatibility Named Pipe. Starting Apply Kernel Variables... [ OK ] Created slice system-sshd\x2dkeygen.slice. [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Reached target Local Encrypted Volumes. [ OK ] Reached target Paths. Activating swap /dev/disk/by-label/SWAP... [ OK ] Created slice User and Session Slice. [ OK ] Reached target Slices. Mounting POSIX Message Queue File System... [ OK ] Created slice[ 25.059603] Adding 1048572k swap on /dev/vdb. Priority:-2 extents:1 across:1048572k FS system-serial\x2dgetty.slice. Mounting Kernel Debug File System... [ OK ] Listening on RPCbind Server Activation Socket. [ OK ] Reached target RPC Port Mapper. [ OK ] Stopped target Initrd Root File System. Starting Create list of required st…ce nodes for the current kernel... [FAILED] Failed to set up automount Arbitrar…rmats File System Automount Point. See 'systemctl status proc-sys-fs-binfmt_misc.automount' for details. [ OK ] Started Journal Service. [FAILED] Failed to start Remount Root and Kernel File Systems. See 'systemctl status systemd-remount-fs.service' for details. [ OK ] Mounted Huge Pages File System. [ OK ] Started Apply Kernel Variables. [ OK ] Activated swap /dev/disk/by-label/SWAP. [ OK ] Mounted POSIX Message Queue File System. [ OK ] Mounted Kernel Debug File System. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Reached target Swap. Starting Configure read-only root support... Starting Create Static Device Nodes in /dev... Starting Flush Journal to Persistent Storage... [ OK ] Started udev Coldplug all Devices. [ OK ] Started Flush Journal to Persistent Storage. [ OK ] Started Create Static Device Nodes in /dev. [ OK ] Reached target Local File Systems (Pre). Mounting /mnt... Mounting /home/green/git/lustre-release... Starting udev Kernel Device Manager... [ OK ] Mounted /mnt. [ 28.462912] squashfs: version 4.0 (2009/01/31) Phillip Lougher [ OK ] Mounted /home/green/git/lustre-release. [ OK ] Started udev Kernel Device Manager. [ 30.351845] piix4_smbus 0000:00:01.3: SMBus Host Controller at 0x700, revision 0 [ 31.108584] input: PC Speaker as /devices/platform/pcspkr/input/input5 [* ] A start job is running for Configur…only root support (12s / no limit) [** ] A start job is running for Configur…only root support (12s / no limit) [*** ] A start job is running for Configur…only root support (13s / no limit) [ *** ] A start job is running for Configur…only root support (13s / no limit) [ *** ] A start job is running for Configur…only root support (14s / no limit) [ ***] A start job is running for Configur…only root support (15s / no limit) [ **] A start job is running for Configur…only root support (15s / no limit) [ *] A start job is running for Configur…only root support (16s / no limit)[ 39.352297] RAPL PMU: API unit is 2^-32 Joules, 0 fixed counters, 10737418240 ms ovfl timer [ 39.432891] EDAC sbridge: Ver: 1.1.2 [ **] A start job is running for Configur…only root support (17s / no limit) [ ***] A start job is running for Configur…only root support (17s / no limit) [ *** ] A start job is running for Configur…only root support (18s / no limit) [ *** ] A start job is running for Configur…only root support (18s / no limit) [*** ] A start job is running for Configur…only root support (19s / no limit) [** ] A start job is running for Configur…only root support (19s / no limit) [* ] A start job is running for Configur…only root support (20s / no limit) [** ] A start job is running for Configur…only root support (20s / no limit)[ 43.557987] Key type dns_resolver registered [*** ] A start job is running for Configur…only root support (21s / no limit)[ 44.023554] NFS: Registering the id_resolver key type [ 44.028044] Key type id_resolver registered [ 44.034960] Key type id_legacy registered [ *** ] A start job is running for Configur…only root support (21s / no limit) [ *** ] A start job is running for Configur…only root support (22s / no limit) [ ***] A start job is running for Configur…only root support (22s / no limit) [ **] A start job is running for Configur…only root support (22s / no limit) [ OK ] Started Configure read-only root support. [ OK ] Reached target Local File Systems. Starting Rebuild Dynamic Linker Cache... Starting Mark the need to relabel after reboot... Starting Load/Save Random Seed... Starting Create Volatile Files and Directories... [ OK ] Started Mark the need to relabel after reboot. [ OK ] Started Load/Save Random Seed. [ OK ] Started Create Volatile Files and Directories. Starting RPC Bind... Starting Update UTMP about System Boot/Shutdown... [ OK ] Started Update UTMP about System Boot/Shutdown. [ OK ] Started RPC Bind. [ *] A start job is running for Rebuild …amic Linker Cache (30s / no limit) [ OK ] Started Rebuild Dynamic Linker Cache. Starting Update is Completed... [ OK ] Started Update is Completed. [ OK ] Reached target System Initialization. [ OK ] Listening on D-Bus System Message Bus Socket. [ OK ] Reached target Sockets. [ OK ] Started Daily Cleanup of Temporary Directories. [ OK ] Started dnf makecache --timer. [ OK ] Reached target Basic System. Starting Login Service... [ OK ] Started Hardware RNG Entropy Gatherer Daemon. Starting Restore /run/initramfs on shutdown... [ OK ] Started irqbalance daemon. [ OK ] Started D-Bus System Message Bus. Starting Network Manager... [ OK ] Reached target sshd-keygen.target. [ OK ] Started daily update of the root trust anchor for DNSSEC. [ OK ] Reached target Timers. [ OK ] Started Restore /run/initramfs on shutdown. [ OK ] Started Login Service. [ OK ] Started Network Manager. Starting Network Manager Wait Online... [ OK ] Reached target Network. Starting GSSAPI Proxy Daemon... Starting OpenSSH server daemon... Starting Dynamic System Tuning Daemon... [ OK ] Started OpenSSH server daemon. Starting Hostname Service... [ OK ] Started GSSAPI Proxy Daemon. [ OK ] Reached target NFS client services. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Starting Permit User Sessions... [ OK ] Started Permit User Sessions. [ OK ] Started Serial Getty on ttyS0. [ OK ] Started Getty on tty1. [ OK ] Started Command Scheduler. [ OK ] Started Serial Getty on ttyS1. [ OK ] Reached target Login Prompts. [ OK ] Started Hostname Service. Starting Network Manager Script Dispatcher Service... [ OK ] Started Network Manager Script Dispatcher Service. [ OK ] Started Network Manager Wait Online. [ OK ] Reached target Network is Online. Starting System Logging Service... Starting Crash recovery kernel arming... Starting Notify NFS peers of a restart... Rocky Linux 8.10 (Green Obsidian) Kernel 4.18.0rh8.10-debug on an x86_64 oleg247-server login: [ 161.898543] libcfs: loading out-of-tree module taints kernel. [ 161.965519] Key type ._llcrypt registered [ 161.967612] Key type .llcrypt registered [ 162.065982] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_hostid [ 180.854859] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing load_modules_local [ 182.497687] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 182.514658] alg: No test for adler32 (adler32-zlib) [ 183.902192] Lustre: Lustre: Build Version: 2.17.57_47_gae0d133 [ 184.873530] LNet: Added LNI 192.168.202.147@tcp [8/256/0/180] [ 186.695214] Key type lgssc registered [ 188.506660] Lustre: Echo OBD driver; http://www.lustre.org/ [ 206.676074] ZFS: Loaded module v2.3.2-1, ZFS pool version 5000, ZFS filesystem version 5 [ 250.630493] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing load_modules_local [ 265.100635] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 265.139547] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 266.498224] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 266.527669] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 266.696975] Lustre: lustre-MDT0000: new disk, initializing [ 266.818047] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 266.842286] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 271.447970] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 285.154945] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 285.382386] Lustre: 6516:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 285.439936] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 285.449901] Lustre: Skipped 1 previous similar message [ 285.564042] Lustre: lustre-MDT0001: new disk, initializing [ 285.644636] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 285.680971] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 285.694566] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 290.866569] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 295.961162] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 306.220353] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 306.586930] Lustre: lustre-OST0000: new disk, initializing [ 306.589703] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 306.593339] Lustre: 8454:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 306.683436] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 309.270139] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 309.284182] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 309.366660] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 313.777906] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 330.400845] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 330.628680] Lustre: lustre-OST0001: new disk, initializing [ 330.631689] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 330.645601] Lustre: 9528:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 330.784796] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 336.482220] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 336.497159] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 336.571995] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 339.147485] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 353.883668] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 365.176221] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 372.780424] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing check_logdir /tmp/testlogs/ [ 378.166611] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing yml_node [ 383.929815] Lustre: DEBUG MARKER: Client: 2.17.57.47 [ 387.211556] Lustre: DEBUG MARKER: MDS: 2.17.57.47 [ 390.898976] Lustre: DEBUG MARKER: OSS: 2.17.57.47 [ 393.468102] Lustre: DEBUG MARKER: -----============= acceptance-small: conf-sanity ============----- Sat Aug 22 00:34:06 EDT 2026 [ 414.720823] Lustre: DEBUG MARKER: excepting tests: 102 106 115 32newtarball [ 415.936758] Lustre: DEBUG MARKER: skipping tests SLOW=no: 45 69 106 111 114 [ 434.145683] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 434.148318] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 434.149499] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 434.154602] Lustre: Skipped 3 previous similar messages [ 434.156317] Lustre: Skipped 2 previous similar messages [ 437.844554] Lustre: server umount lustre-MDT0000 complete [ 443.975264] LustreError: 6509:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787373298 with bad export cookie 9495201595542106185 [ 443.976732] LustreError: MGC192.168.202.147@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 444.010905] LustreError: 6509:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 444.384558] LustreError: 6524:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 444.386545] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 444.388891] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 444.388900] Lustre: Skipped 1 previous similar message [ 444.420515] LustreError: 6524:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 6 previous similar messages [ 444.483766] Lustre: Skipped 1 previous similar message [ 448.417929] LustreError: 6524:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 448.421507] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 448.494429] Lustre: Skipped 1 previous similar message [ 451.431720] Lustre: server umount lustre-MDT0001 complete [ 457.163779] Lustre: server umount lustre-OST0000 complete [ 460.790771] Lustre: server umount lustre-OST0001 complete [ 466.496879] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_hostid [ 477.205687] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing load_modules_local [ 516.240977] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 516.529992] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 516.572712] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 516.748884] Lustre: lustre-MDT0000: new disk, initializing [ 516.918412] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 516.962363] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 522.907583] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 533.761849] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 533.963589] Lustre: 15085:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 534.066782] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 534.070345] Lustre: Skipped 1 previous similar message [ 534.210395] Lustre: lustre-MDT0001: new disk, initializing [ 534.329447] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 534.379775] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 534.394547] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 540.065459] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 550.245634] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 556.871176] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 563.526341] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 563.781738] Lustre: lustre-OST0000: new disk, initializing [ 563.789416] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 563.796845] Lustre: 17057:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 563.895490] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 565.196876] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 565.220448] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 565.374959] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 570.092230] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 578.816348] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 583.408855] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 583.686190] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 588.078353] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 588.236956] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 590.822350] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 590.825060] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 590.834253] LustreError: Skipped 1 previous similar message [ 590.840342] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 595.915187] Lustre: server umount lustre-OST0000 complete [ 601.056825] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 601.058979] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 601.071443] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 601.074821] Lustre: Skipped 2 previous similar messages [ 601.090875] Lustre: Skipped 2 previous similar messages [ 605.806779] Lustre: server umount lustre-MDT0000 complete [ 606.186512] LustreError: 15935:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 606.203967] LustreError: 15935:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 609.716975] LustreError: 15078:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787373464 with bad export cookie 9495201595542107718 [ 609.718364] LustreError: MGC192.168.202.147@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 609.723886] LustreError: 15078:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 610.185345] Lustre: server umount lustre-MDT0001 complete [ 615.133465] Lustre: DEBUG MARKER: == conf-sanity test 76a: set permanent params with lctl across mounts ========================================================== 00:37:49 (1787373469) [ 621.555488] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 621.999752] LustreError: 19321:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 622.067285] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 627.141771] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 627.173383] LustreError: 19322:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 627.195271] LustreError: 19322:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 635.954829] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 641.365636] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 650.615571] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 657.395929] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 663.815318] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 671.826037] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 679.499868] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 686.207919] Lustre: Modifying parameter general.osc.*.max_dirty_mb=452 in log params [ 697.315558] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 697.318357] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 697.333630] Lustre: Skipped 1 previous similar message [ 699.745811] Lustre: server umount lustre-MDT0000 complete [ 702.437944] LustreError: 19322:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 702.461555] LustreError: 19322:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [ 703.582678] LustreError: 19303:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787373558 with bad export cookie 9495201595542108600 [ 703.590696] LustreError: MGC192.168.202.147@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 703.599623] LustreError: 19303:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 704.105653] Lustre: server umount lustre-MDT0001 complete [ 717.596269] Lustre: server umount lustre-OST0000 complete [ 743.492963] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing load_modules_local [ 755.485256] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 755.902560] LustreError: 23703:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 755.983239] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 755.986083] Lustre: Skipped 2 previous similar messages [ 761.259830] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 770.836765] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 776.730773] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 779.835883] Lustre: Setting parameter general.debug_raw_pointers=Y in log params [ 786.066592] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 792.705224] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 801.137559] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 801.299207] Lustre: lustre-OST0001: new disk, initializing [ 801.302221] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 801.308740] Lustre: 26019:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 803.180818] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 803.193488] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 803.319890] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 808.086907] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 818.214821] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 821.493317] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 838.113635] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 838.120433] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 838.145243] Lustre: Skipped 1 previous similar message [ 838.166345] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 838.169496] Lustre: Skipped 2 previous similar messages [ 840.363223] Lustre: server umount lustre-MDT0000 complete [ 844.140312] LustreError: 23684:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787373699 with bad export cookie 9495201595542109440 [ 844.142389] LustreError: MGC192.168.202.147@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 844.155126] LustreError: 23684:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 844.265543] LustreError: 23698:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 844.285670] LustreError: 23698:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 4 previous similar messages [ 844.686622] Lustre: server umount lustre-MDT0001 complete [ 850.460886] Lustre: server umount lustre-OST0000 complete [ 854.387823] Lustre: server umount lustre-OST0001 complete [ 870.151497] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing load_modules_local [ 882.195648] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 888.305161] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 896.676977] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 896.934566] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 896.939849] Lustre: Skipped 4 previous similar messages [ 902.084899] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 905.116806] Lustre: 30187:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 911.984412] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 919.198886] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 921.634385] LustreError: 30543:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 921.663654] LustreError: 30543:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [ 927.894160] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 935.564251] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 943.206916] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 946.538695] Lustre: 32067:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 953.312425] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 953.323061] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 953.338911] Lustre: Skipped 5 previous similar messages [ 953.350891] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 953.363031] Lustre: Skipped 6 previous similar messages [ 958.871407] Lustre: server umount lustre-MDT0000 complete [ 962.587570] LustreError: 30190:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787373817 with bad export cookie 9495201595542110742 [ 962.596099] LustreError: MGC192.168.202.147@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 962.609939] LustreError: 30190:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 963.078710] Lustre: server umount lustre-MDT0001 complete [ 977.182398] Lustre: server umount lustre-OST0000 complete [ 979.423137] Lustre: 3649:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787373818/real 1787373819] req@ffff9ffd84728e00 x1874196472043520/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1787373834 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 980.999784] Lustre: server umount lustre-OST0001 complete [ 990.125450] Lustre: DEBUG MARKER: == conf-sanity test 76b: verify params log setup correctly ========================================================== 00:44:04 (1787373844) [ 1015.838208] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing load_modules_local [ 1026.481408] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1031.581628] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 1040.452727] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1044.832829] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 1048.116969] Lustre: 36016:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1055.152952] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1064.010595] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 1066.027611] LustreError: 36369:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1066.052887] LustreError: 36369:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 10 previous similar messages [ 1072.231062] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1077.857812] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 1085.369427] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1089.076454] Lustre: 37899:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 1097.186534] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1097.195055] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1097.217100] Lustre: Skipped 4 previous similar messages [ 1097.228750] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1097.235630] Lustre: Skipped 3 previous similar messages [ 1099.448789] Lustre: server umount lustre-MDT0000 complete [ 1102.990612] LustreError: 34850:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787373957 with bad export cookie 9495201595542111848 [ 1102.993517] LustreError: MGC192.168.202.147@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1102.999527] LustreError: 34850:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1103.305147] Lustre: server umount lustre-MDT0001 complete [ 1116.315354] Lustre: server umount lustre-OST0000 complete [ 1119.329090] Lustre: 3650:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787373958/real 1787373958] req@ffff9ffd84a4bb80 x1874196472092416/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1787373974 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1120.339642] Lustre: server umount lustre-OST0001 complete [ 1129.072993] Lustre: DEBUG MARKER: == conf-sanity test 76c: verify changelog_mask is applied with lctl set_param -P ========================================================== 00:46:22 (1787373982) [ 1145.890477] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing load_modules_local [ 1156.555953] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1156.941519] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1156.950271] Lustre: Skipped 6 previous similar messages [ 1162.211250] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 1171.656719] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1175.971279] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 1178.523858] Lustre: 41186:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1184.773536] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1191.078930] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 1199.603387] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1205.837413] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 1212.988152] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1216.487438] Lustre: 43065:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 1217.276896] Lustre: Setting parameter general.mdd.*.changelog_mask=-CLOSE in log params [ 1217.289813] Lustre: Skipped 1 previous similar message [ 1222.627151] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1222.659559] Lustre: Skipped 5 previous similar messages [ 1226.234175] Lustre: server umount lustre-MDT0000 complete [ 1229.751462] LustreError: 40018:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787374084 with bad export cookie 9495201595542112954 [ 1229.763566] LustreError: MGC192.168.202.147@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1232.872686] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 1232.878200] Lustre: Skipped 10 previous similar messages [ 1244.127191] Lustre: lustre-MDT0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 1244.392875] Lustre: server umount lustre-MDT0001 complete [ 1252.207145] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1254.303148] LustreError: 43927:0:(import.c:339:ptlrpc_invalidate_import()) MGS: timeout waiting for callback (1 != 0) [ 1254.315671] LustreError: 43927:0:(import.c:363:ptlrpc_invalidate_import()) @@@ still on sending list req@ffff9ffc830c1880 x1874196472141824/t0(0) o250->MGC192.168.202.147@tcp@0@lo:26/25 lens 520/544 e 0 to 0 dl 1787374109 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1254.357521] LustreError: 43927:0:(import.c:373:ptlrpc_invalidate_import()) MGS: Unregistering RPCs found (0). Network is sluggish? Waiting for them to error out. [ 1254.382141] LustreError: 3647:0:(client.c:1404:ptlrpc_import_delay_req()) @@@ invalidate in flight req@ffff9ffda651d180 x1874196472142976/t0(0) o250->MGC192.168.202.147@tcp@0@lo:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl Rpc:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1259.555225] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 1268.792182] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1270.212692] Lustre: lustre-MDT0000-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 1273.981184] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 1281.043932] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1282.651885] Lustre: DEBUG MARKER: mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec [ 1288.400361] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 1289.964866] Lustre: DEBUG MARKER: mdc.lustre-MDT0001-mdc-*.mds_server_uuid in FULL state after 0 sec [ 1294.817987] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1301.101385] Lustre: server umount lustre-MDT0000 complete [ 1304.727379] LustreError: 40788:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787374159 with bad export cookie 9495201595542114053 [ 1304.737119] LustreError: MGC192.168.202.147@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1304.743666] LustreError: 40788:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 8 previous similar messages [ 1305.053942] Lustre: server umount lustre-MDT0001 complete [ 1318.784572] Lustre: server umount lustre-OST0000 complete [ 1332.851804] Lustre: server umount lustre-OST0001 complete [ 1340.313713] Lustre: DEBUG MARKER: == conf-sanity test 76d: verify llite.*.xattr_cache can be set by 'lctl set_param -P' correctly ========================================================== 00:49:54 (1787374194) [ 1353.464858] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing load_modules_local [ 1362.053382] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1362.370586] LustreError: 47673:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1362.381625] LustreError: 47673:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 30 previous similar messages [ 1366.373310] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 1374.147683] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1379.486821] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 1382.218405] Lustre: 48825:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1388.121371] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1394.952773] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 1403.159383] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1409.323538] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 1416.556858] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1420.783346] Lustre: Setting parameter general.llite.*.xattr_cache=0 in log params [ 1431.008598] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1431.019180] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1431.035985] Lustre: Skipped 8 previous similar messages [ 1432.508250] Lustre: server umount lustre-MDT0000 complete [ 1436.148881] LustreError: 47653:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787374291 with bad export cookie 9495201595542114872 [ 1436.154649] LustreError: MGC192.168.202.147@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1436.172695] LustreError: 47653:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1436.534248] Lustre: server umount lustre-MDT0001 complete [ 1451.122533] Lustre: server umount lustre-OST0000 complete [ 1465.169264] Lustre: server umount lustre-OST0001 complete [ 1474.472575] Lustre: DEBUG MARKER: == conf-sanity test 77: comma-separated MGS NIDs and failover node NIDs ========================================================== 00:52:08 (1787374328) [ 1476.428190] Lustre: DEBUG MARKER: SKIP: conf-sanity test_77 mixed loopback and real device not working [ 1478.200251] Lustre: DEBUG MARKER: == conf-sanity test 78: run resize2fs on MDT and OST filesystems ========================================================== 00:52:12 (1787374332) [ 1488.284404] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_hostid [ 1496.417047] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing load_modules_local [ 1516.852234] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1517.061380] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1517.144381] Lustre: lustre-MDT0000: new disk, initializing [ 1517.228382] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1521.770929] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 1530.029826] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1536.051704] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1536.348445] Lustre: lustre-OST0000: new disk, initializing [ 1536.351857] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1536.358718] Lustre: Skipped 1 previous similar message [ 1536.364235] Lustre: 54901:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1537.976277] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 1537.985275] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 1538.023557] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 1542.778402] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 1551.538807] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1553.722832] Lustre: DEBUG MARKER: create test files [ 1584.096365] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1584.107959] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1584.118804] Lustre: Skipped 13 previous similar messages [ 1585.826517] Lustre: server umount lustre-OST0000 complete [ 1589.680147] Lustre: server umount lustre-MDT0000 complete [ 1600.094939] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing unload_modules_local [ 1602.504525] Key type lgssc unregistered [ 1602.776481] LNet: 56535:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1602.783160] LNetError: 56535:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1602.798371] LNet: Removed LNI 192.168.202.147@tcp [ 1603.834160] Key type .llcrypt unregistered [ 1603.841028] Key type ._llcrypt unregistered [ 1631.790285] Key type ._llcrypt registered [ 1631.792935] Key type .llcrypt registered [ 1631.920545] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing load_modules_local [ 1633.100082] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1633.339180] alg: No test for adler32 (adler32-zlib) [ 1634.362697] Lustre: Lustre: Build Version: 2.17.57_47_gae0d133 [ 1634.639780] LNet: Added LNI 192.168.202.147@tcp [8/256/0/180] [ 1636.319265] Key type lgssc registered [ 1637.232662] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1643.531732] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1643.565746] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1645.039613] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1649.913670] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 1655.929951] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1661.047700] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1661.253411] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1666.142119] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 1669.439938] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:102 to 0x240000400:129) [ 1669.569671] Lustre: 59265:0:(osd_internal.h:1460:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 511 < left 544, rollback = 9 [ 1669.581540] Lustre: 59265:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 32/128/4, destroy: 0/0/0 [ 1669.586865] Lustre: 59265:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 1669.597371] Lustre: 59265:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/18/1, punch: 0/0/0, quota 1/3/0 [ 1669.605595] Lustre: 59265:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 1669.614736] Lustre: 59265:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 1672.708859] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1678.323658] Lustre: DEBUG MARKER: check files after expanding the MDT and OST filesystems [ 1682.991724] Lustre: DEBUG MARKER: create more files after expanding the MDT and OST filesystems [ 1690.087930] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1690.105469] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1690.109803] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1691.517074] Lustre: server umount lustre-OST0000 complete [ 1695.473963] Lustre: server umount lustre-MDT0000 complete [ 1706.097306] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing unload_modules_local [ 1708.610600] Key type lgssc unregistered [ 1708.896844] LNet: 60878:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1708.902282] LNetError: 60878:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1708.919925] LNet: Removed LNI 192.168.202.147@tcp [ 1709.679487] Key type .llcrypt unregistered [ 1709.681307] Key type ._llcrypt unregistered [ 1735.660558] Key type ._llcrypt registered [ 1735.662641] Key type .llcrypt registered [ 1735.765887] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing load_modules_local [ 1737.001943] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1737.043156] alg: No test for adler32 (adler32-zlib) [ 1738.199874] Lustre: Lustre: Build Version: 2.17.57_47_gae0d133 [ 1738.538187] LNet: Added LNI 192.168.202.147@tcp [8/256/0/180] [ 1740.167179] Key type lgssc registered [ 1740.995544] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1747.914583] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1747.935418] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1749.413474] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1754.086624] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 1760.596332] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1767.014968] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1767.417215] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1773.058954] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 1774.603293] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:140 to 0x240000400:161) [ 1774.662335] Lustre: 63690:0:(osd_internal.h:1460:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 509 < left 544, rollback = 9 [ 1774.671872] Lustre: 63690:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 32/128/6, destroy: 0/0/0 [ 1774.681484] Lustre: 63690:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 1774.690366] Lustre: 63690:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/18/1, punch: 0/0/0, quota 1/3/0 [ 1774.698750] Lustre: 63690:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 1774.706640] Lustre: 63690:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 1779.703824] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1784.637754] Lustre: DEBUG MARKER: check files after shrinking the MDT and OST filesystems [ 1825.773369] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1825.778735] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1825.794644] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1827.904570] Lustre: server umount lustre-OST0000 complete [ 1831.445767] Lustre: server umount lustre-MDT0000 complete [ 1842.591914] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing unload_modules_local [ 1845.473357] Key type lgssc unregistered [ 1845.819918] LNet: 65277:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1845.858873] LNetError: 65277:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1845.881276] LNet: Removed LNI 192.168.202.147@tcp [ 1846.991849] Key type .llcrypt unregistered [ 1846.997788] Key type ._llcrypt unregistered [ 1864.817436] Key type ._llcrypt registered [ 1864.819586] Key type .llcrypt registered [ 1864.910310] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_hostid [ 1884.742626] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing load_modules_local [ 1886.476611] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1886.688297] alg: No test for adler32 (adler32-zlib) [ 1887.714147] Lustre: Lustre: Build Version: 2.17.57_47_gae0d133 [ 1887.990469] LNet: Added LNI 192.168.202.147@tcp [8/256/0/180] [ 1889.775363] Key type lgssc registered [ 1891.028390] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1924.565625] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1924.600567] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1925.967910] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 1926.016544] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1926.077370] Lustre: lustre-MDT0000: new disk, initializing [ 1926.130843] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1926.151349] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1930.281693] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 1941.122625] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1941.275490] Lustre: 68434:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 1941.381259] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 1941.393834] Lustre: Skipped 1 previous similar message [ 1941.400774] Lustre: lustre-MDT0001: Not available for connect from 0@lo (not set up) [ 1941.569889] Lustre: lustre-MDT0001: new disk, initializing [ 1941.714720] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 1941.798636] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 1941.812298] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 1946.219379] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 1956.397294] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1962.293153] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 1969.680380] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1969.975944] Lustre: lustre-OST0000: new disk, initializing [ 1969.981440] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1969.994401] Lustre: 70400:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1970.147678] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1977.076824] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 1980.102661] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 1980.118703] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 1980.282110] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 1987.782715] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1991.699373] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 1991.907659] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 1996.664148] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 1996.871884] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2000.894469] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2000.894960] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2000.902471] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2000.905622] LustreError: Skipped 1 previous similar message [ 2000.943693] Lustre: Skipped 1 previous similar message [ 2004.952564] Lustre: server umount lustre-OST0000 complete [ 2011.105331] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2011.131738] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2011.136895] Lustre: Skipped 1 previous similar message [ 2016.226270] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2016.235598] Lustre: Skipped 1 previous similar message [ 2020.321129] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2020.327457] Lustre: Skipped 1 previous similar message [ 2023.392975] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2023.678669] Lustre: server umount lustre-MDT0000 complete [ 2027.240067] LustreError: 68424:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787374882 with bad export cookie 9947003206696756377 [ 2027.247195] LustreError: MGC192.168.202.147@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2027.250238] LustreError: 68424:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2027.691889] Lustre: server umount lustre-MDT0001 complete [ 2035.452235] Lustre: DEBUG MARKER: == conf-sanity test 79: format MDT/OST without mgs option (should return errors) ========================================================== 01:01:29 (1787374889) [ 2045.682833] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing load_modules_local [ 2083.873651] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_hostid [ 2094.663263] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing load_modules_local [ 2133.943185] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2134.246035] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2134.279143] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2134.419878] Lustre: lustre-MDT0000: new disk, initializing [ 2134.537121] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2134.601112] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2140.325398] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 2151.235485] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2151.369453] Lustre: 76434:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2151.447305] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 2151.453962] Lustre: Skipped 1 previous similar message [ 2151.575942] Lustre: lustre-MDT0001: new disk, initializing [ 2151.652710] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2151.684959] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 2151.698982] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2156.500824] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 2165.809596] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2171.831352] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2178.697072] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2178.925481] Lustre: lustre-OST0000: new disk, initializing [ 2178.929588] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 2178.935700] Lustre: 78405:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2179.038875] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2185.153356] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 2185.300689] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 2185.311174] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2185.413273] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2193.558901] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2198.819440] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2199.138046] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2204.243440] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2204.421110] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2206.177508] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2206.185474] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2206.186779] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2206.193667] Lustre: Skipped 2 previous similar messages [ 2206.203622] Lustre: Skipped 1 previous similar message [ 2212.589738] Lustre: server umount lustre-OST0000 complete [ 2218.465186] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2218.480987] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2222.875049] Lustre: server umount lustre-MDT0000 complete [ 2226.539154] LustreError: 76426:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787375081 with bad export cookie 9947003206696757266 [ 2226.546483] LustreError: 76426:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 2226.548607] LustreError: MGC192.168.202.147@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2226.856404] Lustre: server umount lustre-MDT0001 complete [ 2234.211490] Lustre: DEBUG MARKER: == conf-sanity test 80: mgc import reconnect race ======== 01:04:48 (1787375088) [ 2240.124377] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2240.610702] LustreError: 80816:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2240.626655] LustreError: 80816:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [ 2240.697043] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2244.973187] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 2246.115458] LustreError: 80817:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2250.217365] LustreError: 80816:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2253.517038] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2258.079249] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 2265.439028] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2272.506392] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2279.091518] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2279.451335] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2279.458978] Lustre: Skipped 1 previous similar message [ 2285.836776] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 2292.477974] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2295.590955] Lustre: 83487:0:(genops.c:1774:obd_export_evict_by_uuid()) MGS: evicting 90ca5cd0-c238-4165-911d-9799db2f371b at adminstrative request [ 2296.803201] LustreError: MGC192.168.202.147@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2296.820624] Lustre: *** cfs_fail_loc=906, val=2147483648*** [ 2307.048245] Lustre: MGS: Client 90ca5cd0-c238-4165-911d-9799db2f371b (at 0@lo) reconnecting [ 2307.052487] Lustre: Evicted from MGS (at 0@lo) after server handle changed from 0x8a0adab70ad4eb84 to 0x8a0adab70ad4ee1d [ 2307.057633] Lustre: MGC192.168.202.147@tcp: Connection restored to 0@lo (at 0@lo) [ 2334.518430] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2334.786248] Lustre: lustre-OST0001: new disk, initializing [ 2334.792254] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 2334.800146] Lustre: 83883:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2340.524757] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 2340.536121] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 2340.662919] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 2340.909832] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 2350.367335] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2356.202751] LustreError: lustre-OST0001-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2356.204716] Lustre: lustre-OST0001-osc-MDT0000: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2356.206529] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 2356.206537] Lustre: Skipped 5 previous similar messages [ 2356.252760] Lustre: Skipped 2 previous similar messages [ 2359.105020] Lustre: server umount lustre-OST0001 complete [ 2361.322495] LustreError: 84214:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2361.348626] LustreError: 84214:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 2363.364025] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2363.373016] LustreError: Skipped 1 previous similar message [ 2365.416895] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2365.431165] Lustre: Skipped 1 previous similar message [ 2366.448857] LustreError: 84214:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2366.460845] LustreError: 84214:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 2376.679467] LustreError: 84216:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2376.689896] LustreError: 84216:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [ 2377.183146] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2377.393557] Lustre: server umount lustre-OST0000 complete [ 2381.793914] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2381.795727] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2381.806815] Lustre: Skipped 1 previous similar message [ 2381.833253] LustreError: Skipped 1 previous similar message [ 2392.041556] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2392.047142] Lustre: Skipped 14 previous similar messages [ 2395.103156] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2395.315569] Lustre: server umount lustre-MDT0000 complete [ 2397.153094] LustreError: 80817:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2398.913781] LustreError: 80799:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787375253 with bad export cookie 9947003206696758813 [ 2398.921701] LustreError: MGC192.168.202.147@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2398.925638] LustreError: 80799:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 2399.250755] Lustre: server umount lustre-MDT0001 complete [ 2407.043383] Lustre: DEBUG MARKER: == conf-sanity test 81: sparse OST indexing ============== 01:07:41 (1787375261) [ 2408.539870] Lustre: DEBUG MARKER: SKIP: conf-sanity test_81 needs >= 3 OSTs [ 2410.156047] Lustre: DEBUG MARKER: == conf-sanity test 82a: specify OSTs for file (succeed) or directory (succeed) ========================================================== 01:07:44 (1787375264) [ 2411.839939] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82a needs >= 3 OSTs [ 2413.433335] Lustre: DEBUG MARKER: == conf-sanity test 82b: specify OSTs for file with --pool and --ost-list options ========================================================== 01:07:47 (1787375267) [ 2414.976306] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82b needs >= 4 OSTs [ 2416.609450] Lustre: DEBUG MARKER: == conf-sanity test 82c: specify sparse OSTs for setstripe (should not crash) ========================================================== 01:07:50 (1787375270) [ 2418.183620] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82c needs >= 4 OSTs [ 2420.051669] Lustre: DEBUG MARKER: == conf-sanity test 83: ENOSPACE on OST doesn't cause message VFS: Busy inodes after unmount ... ========================================================== 01:07:53 (1787375273) [ 2421.534631] Lustre: DEBUG MARKER: mount the OST /dev/mapper/ost1_flakey as a ldiskfs filesystem [ 2427.687327] loop: module loaded [ 2427.696253] loop0: detected capacity change from 0 to 4294967296 [ 2427.725039] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 2429.365551] Lustre: DEBUG MARKER: run llverfs in partial mode on the OST ldiskfs /mnt/lustre-ost1 [ 2433.020713] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing run_llverfs /mnt/lustre-ost1 -vpl no [ 2437.350034] Lustre: DEBUG MARKER: unmount the OST /dev/mapper/ost1_flakey [ 2464.919118] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2465.000428] LustreError: 88009:0:(obd_config.c:845:class_setup()) setup lustre-OST0000-osd failed (-28) [ 2465.011896] LustreError: 88009:0:(obd_mount.c:259:lustre_start_simple()) lustre-OST0000-osd setup error -28 [ 2465.021246] LustreError: 88009:0:(tgt_mount.c:2579:server_fill_super()) Unable to start osd on /dev/mapper/ost1_flakey: -28 [ 2465.036056] LustreError: 88009:0:(super25.c:179:lustre_fill_super()) llite: Unable to mount : rc = -28 [ 2485.116223] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_hostid [ 2496.342165] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing load_modules_local [ 2530.616092] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2530.841466] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2530.869261] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2530.954983] Lustre: lustre-MDT0000: new disk, initializing [ 2531.041615] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2531.048965] Lustre: Skipped 1 previous similar message [ 2531.074315] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2535.512576] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 2545.814201] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2545.926711] Lustre: 90872:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2546.164335] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2550.845240] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 2561.048965] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2567.116330] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2574.149330] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2574.421284] Lustre: 92842:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2580.417259] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 2580.570019] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2580.788775] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2590.416973] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2595.541504] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2595.826577] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2602.015669] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2602.314322] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2606.563097] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2606.572251] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2606.588971] LustreError: Skipped 1 previous similar message [ 2606.592110] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2606.616941] Lustre: Skipped 1 previous similar message [ 2606.655759] Lustre: Skipped 1 previous similar message [ 2610.677605] Lustre: server umount lustre-OST0000 complete [ 2621.065817] Lustre: server umount lustre-MDT0000 complete [ 2621.921143] LustreError: 90886:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2621.944687] LustreError: 90886:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 2625.251960] LustreError: 90865:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787375480 with bad export cookie 9947003206696759436 [ 2625.253417] LustreError: MGC192.168.202.147@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2625.264533] LustreError: 90865:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 2625.634701] Lustre: server umount lustre-MDT0001 complete [ 2634.775664] Lustre: DEBUG MARKER: == conf-sanity test 84: check recovery_hard_time ========= 01:11:28 (1787375488) [ 2642.538891] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2647.396168] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 2655.235617] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2659.732163] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 2666.716573] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2672.020580] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2678.267164] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2678.543749] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2678.548657] Lustre: Skipped 4 previous similar messages [ 2685.088334] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 2691.776375] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2695.707872] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2695.853918] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2699.388798] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2699.652482] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2705.646967] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2705.863437] Lustre: lustre-OST0001: new disk, initializing [ 2705.870568] Lustre: Skipped 2 previous similar messages [ 2705.878205] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 2705.882738] Lustre: Skipped 3 previous similar messages [ 2705.888254] Lustre: 98623:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2712.432719] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 2714.234732] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 2714.248124] Lustre: Skipped 2 previous similar messages [ 2714.251362] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 2714.314618] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 2721.637898] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2726.102819] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid 50 [ 2726.264798] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2730.643368] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [ 2730.850000] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2748.275356] Lustre: DEBUG MARKER: mds1 REPLAY BARRIER on lustre-MDT0000 [ 2767.214108] Lustre: Failing over lustre-MDT0000 [ 2767.685146] Lustre: server umount lustre-MDT0000 complete [ 2768.352484] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2768.357128] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2768.370437] Lustre: Skipped 2 previous similar messages [ 2768.373875] LustreError: 95258:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2768.385508] LustreError: 95258:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [ 2786.273219] Lustre: 66658:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787375625/real 1787375625] req@0000000078247be3 x1874198257736320/t0(0) o400->MGC192.168.202.147@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1787375641 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 2786.307199] LustreError: MGC192.168.202.147@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2791.874722] LDISKFS-fs (dm-0): 4 truncates cleaned up [ 2791.878687] LDISKFS-fs (dm-0): recovery complete [ 2791.887372] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2796.887770] Lustre: lustre-MDT0000: in recovery but waiting for the first client to connect [ 2800.669054] Lustre: lustre-MDT0000: Will be in recovery for at least 1:00, or until 3 clients reconnect [ 2800.671697] Lustre: 97212:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 2801.767971] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 2802.155475] Lustre: lustre-MDT0000-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 2802.157542] Lustre: 95258:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 2802.183491] Lustre: 95258:0:(ldlm_lib.c:2110:extend_recovery_timer()) Skipped 1 previous similar message [ 2802.321087] LustreError: 101472:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2802.647066] LustreError: 101472:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2802.988936] LustreError: 101472:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2802.997332] LustreError: 101472:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 1 previous similar message [ 2803.329478] LustreError: 101472:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2803.342836] LustreError: 101472:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 1 previous similar message [ 2803.370141] Lustre: 101472:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2803.385338] Lustre: 101472:0:(ldlm_lib.c:2110:extend_recovery_timer()) Skipped 7 previous similar messages [ 2804.046076] LustreError: 101472:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2804.054097] LustreError: 101472:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 2 previous similar messages [ 2804.383151] LustreError: 101472:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2804.393459] LustreError: 101472:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 2 previous similar messages [ 2805.384420] Lustre: 101472:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2805.407849] Lustre: 101472:0:(ldlm_lib.c:2110:extend_recovery_timer()) Skipped 5 previous similar messages [ 2806.048104] LustreError: 101472:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2806.062428] LustreError: 101472:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 5 previous similar messages [ 2806.703133] LustreError: 101472:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2806.722651] LustreError: 101472:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 6 previous similar messages [ 2809.620202] Lustre: 101472:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2809.641349] Lustre: 101472:0:(ldlm_lib.c:2110:extend_recovery_timer()) Skipped 12 previous similar messages [ 2810.289779] LustreError: 101472:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2810.315878] LustreError: 101472:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 12 previous similar messages [ 2810.959832] LustreError: 101472:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2810.966088] LustreError: 101472:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 12 previous similar messages [ 2817.631947] Lustre: 101472:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2817.643780] Lustre: 101472:0:(ldlm_lib.c:2110:extend_recovery_timer()) Skipped 24 previous similar messages [ 2818.290367] LustreError: 101472:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2818.305471] LustreError: 101472:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 24 previous similar messages [ 2819.255187] LustreError: 101472:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2819.263194] LustreError: 101472:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 25 previous similar messages [ 2833.838918] Lustre: 101472:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2833.850991] Lustre: 101472:0:(ldlm_lib.c:2110:extend_recovery_timer()) Skipped 50 previous similar messages [ 2834.497990] LustreError: 101472:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2834.517564] LustreError: 101472:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 50 previous similar messages [ 2835.479183] LustreError: 101472:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2835.493944] LustreError: 101472:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 50 previous similar messages [ 2860.263265] Lustre: 101472:0:(ldlm_lib.c:2419:target_recovery_overseer()) lustre-MDT0000 recovery is aborted by hard timeout [ 2860.268031] Lustre: 101472:0:(ldlm_lib.c:2429:target_recovery_overseer()) recovery is aborted, evict exports in recovery [ 2860.282253] Lustre: 101472:0:(genops.c:1601:class_disconnect_stale_exports()) lustre-MDT0000: disconnect stale client e713fca0-c658-4588-8277-12b090c98036@192.168.202.47@tcp [ 2860.296594] Lustre: lustre-MDT0000: disconnecting 1 stale clients [ 2860.304465] Lustre: 101472:0:(ldlm_lib.c:1939:abort_req_replay_queue()) @@@ aborted: req@000000007aa12240 x1874198237614592/t0(8589934778) o35->e713fca0-c658-4588-8277-12b090c98036@192.168.202.47@tcp:50/0 lens 392/0 e 0 to 0 dl 1787375725 ref 1 fl Complete:/604/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:0 [ 2860.352783] Lustre: lustre-MDT0000-osd: cancel update llog [0x200000400:0x1:0x0] [ 2860.373128] Lustre: lustre-MDT0000: Denying connection for new client e713fca0-c658-4588-8277-12b090c98036 (at 192.168.202.47@tcp), waiting for 3 known clients (2 recovered, 0 in progress, and 1 evicted) to recover in 0:00 [ 2860.399799] Lustre: lustre-MDT0001-osp-MDT0000: cancel update llog [0x240000401:0x1:0x0] [ 2860.423209] Lustre: lustre-MDT0000-osp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 2860.429267] Lustre: lustre-MDT0000: Recovery over after 1:00, of 3 clients 2 recovered and 1 was evicted. [ 2860.431995] Lustre: Skipped 2 previous similar messages [ 2860.478848] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:48 to 0x2c0000401:545) [ 2860.479170] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:79 to 0x280000401:577) [ 2878.959981] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2878.963884] Lustre: Skipped 2 previous similar messages [ 2891.231132] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2891.409562] Lustre: server umount lustre-OST0000 complete [ 2896.872913] LustreError: lustre-OST0001-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2896.882442] Lustre: lustre-OST0001-osc-MDT0000: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2896.897036] Lustre: Skipped 5 previous similar messages [ 2899.428275] LustreError: 97078:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2899.481066] LustreError: 97078:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 41 previous similar messages [ 2901.463927] Lustre: server umount lustre-OST0001 complete [ 2910.916966] Lustre: server umount lustre-MDT0000 complete [ 2914.185805] LustreError: 98622:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787375769 with bad export cookie 9947003206696810655 [ 2914.187065] LustreError: MGC192.168.202.147@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2914.192818] LustreError: 98622:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 2914.416920] Lustre: server umount lustre-MDT0001 complete [ 2922.298333] Lustre: DEBUG MARKER: == conf-sanity test 85: osd_ost init: fail ea_fid_set ==== 01:16:16 (1787375776) [ 2930.351236] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2930.361737] Lustre: *** cfs_fail_loc=197, val=0*** [ 2945.953403] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 2951.072118] LustreError: 103446:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.202.147@tcp: failed processing log, type 4: rc = -110 [ 2956.283807] Lustre: *** cfs_fail_loc=197, val=0*** [ 2956.289254] Lustre: Skipped 65 previous similar messages [ 2976.865428] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2976.871475] Lustre: Skipped 2 previous similar messages [ 2983.166975] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 2990.130411] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2992.522702] Lustre: server umount lustre-OST0000 complete [ 3002.141834] Lustre: DEBUG MARKER: == conf-sanity test 86: Replacing mkfs.lustre -G option == 01:17:35 (1787375855) [ 3025.313791] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3031.129340] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 3041.405792] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3045.830937] LustreError: 105098:0:(osp_object.c:618:osp_attr_get()) lustre-MDT0001-osp-MDT0000: osp_attr_get update error [0x200000009:0x1:0x0]: rc = -5 [ 3045.848339] LustreError: 105098:0:(lod_sub_object.c:917:lod_sub_prep_llog()) lustre-MDT0000-mdtlov: can't get id from catalogs: rc = -5 [ 3045.860588] LustreError: 105098:0:(lod_dev.c:511:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 19, retries 0, failed: rc = -5 [ 3046.281714] Lustre: server umount lustre-MDT0000 complete [ 3080.328636] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_hostid [ 3092.132133] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing load_modules_local [ 3135.327742] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3135.529912] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3135.555571] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3135.639957] Lustre: lustre-MDT0000: new disk, initializing [ 3135.744838] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3140.530301] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 3151.017543] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3151.124595] Lustre: 109274:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 3151.486239] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3155.532314] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 3166.265298] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3174.152365] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3183.120930] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3183.591664] Lustre: 111242:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3190.337321] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 3192.065045] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3200.745597] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3205.879142] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 3206.116433] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 3211.032401] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 3211.239994] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 3217.906601] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3217.910496] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3217.941122] Lustre: Skipped 3 previous similar messages [ 3217.946714] LustreError: Skipped 2 previous similar messages [ 3217.960368] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3217.973786] Lustre: Skipped 13 previous similar messages [ 3219.517883] Lustre: server umount lustre-OST0000 complete [ 3230.147697] Lustre: server umount lustre-MDT0000 complete [ 3233.248523] LustreError: 109285:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3233.290485] LustreError: 109285:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 6 previous similar messages [ 3234.820373] LustreError: 109266:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787376089 with bad export cookie 9947003206696821330 [ 3234.824595] LustreError: MGC192.168.202.147@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3234.844678] LustreError: 109266:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 3235.465483] Lustre: server umount lustre-MDT0001 complete [ 3244.849204] Lustre: DEBUG MARKER: == conf-sanity test 87: check if MDT inode can hold EAs with N stripes properly ========================================================== 01:21:38 (1787376098) [ 3255.102744] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing unload_modules_local [ 3258.403710] Key type lgssc unregistered [ 3258.637987] LNet: 113735:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3258.643836] LNetError: 113735:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3259.685301] LNet: Removed LNI 192.168.202.147@tcp [ 3260.557355] Key type .llcrypt unregistered [ 3260.559818] Key type ._llcrypt unregistered [ 3279.231615] Key type ._llcrypt registered [ 3279.234214] Key type .llcrypt registered [ 3279.363975] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_hostid [ 3295.138689] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing load_modules_local [ 3296.402602] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3296.610174] alg: No test for adler32 (adler32-zlib) [ 3297.632733] Lustre: Lustre: Build Version: 2.17.57_47_gae0d133 [ 3297.863433] LNet: Added LNI 192.168.202.147@tcp [8/256/0/180] [ 3299.519769] Key type lgssc registered [ 3301.223521] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3343.589877] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3343.626092] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3345.188808] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3345.257459] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3345.486706] Lustre: lustre-MDT0000: new disk, initializing [ 3345.619375] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3345.635600] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3350.801852] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 3363.396986] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3363.635418] Lustre: lustre-OST0000: new disk, initializing [ 3363.638170] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3363.644833] Lustre: Skipped 1 previous similar message [ 3363.651371] Lustre: 117912:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3363.702846] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3370.342890] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 3371.123798] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 3371.135666] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 3371.156943] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 3381.036652] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3381.249501] Lustre: lustre-OST0001: new disk, initializing [ 3381.254117] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 3381.263881] Lustre: 118775:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3381.397616] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 3387.038233] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:1:ost [ 3387.050450] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:1:ost] [ 3387.077030] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x280000400 [ 3387.934808] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 3397.605159] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3397.613172] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3397.622147] Lustre: Skipped 1 previous similar message [ 3397.643976] Lustre: Skipped 1 previous similar message [ 3402.721826] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3402.728640] Lustre: Skipped 1 previous similar message [ 3403.521634] Lustre: server umount lustre-MDT0000 complete [ 3410.068203] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: (null) [ 3414.271266] LustreError: 118774:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787376269 with bad export cookie 17159302697729956442 [ 3414.274695] LustreError: MGC192.168.202.147@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3414.276804] LustreError: 118774:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 3414.348286] Lustre: server umount lustre-OST0000 complete [ 3418.073516] Lustre: server umount lustre-OST0001 complete [ 3427.982203] Lustre: DEBUG MARKER: == conf-sanity test 88a: tunefs.lustre persists a --mountfsoptions-only change ========================================================== 01:24:41 (1787376281) [ 3453.339641] Lustre: DEBUG MARKER: == conf-sanity test 88b: check the default mount options can be overridden ========================================================== 01:25:07 (1787376307) [ 3465.793794] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing load_modules_local [ 3489.793281] Lustre: DEBUG MARKER: == conf-sanity test 89: check tunefs --param and --erase-param{s} options ========================================================== 01:25:43 (1787376343) [ 3533.678798] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_hostid [ 3545.090154] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing load_modules_local [ 3585.076844] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3585.644395] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3585.705963] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3585.903215] Lustre: lustre-MDT0000: new disk, initializing [ 3586.130677] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3586.156327] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3591.708203] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 3603.648462] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3603.789594] Lustre: 126535:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 3603.856676] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 3603.859724] Lustre: Skipped 1 previous similar message [ 3603.910683] Lustre: lustre-MDT0001: new disk, initializing [ 3603.964208] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3603.989198] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 3604.000415] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3609.104381] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 3619.400125] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3625.161253] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3633.513692] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3633.775337] Lustre: lustre-OST0000: new disk, initializing [ 3633.780586] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3633.792504] Lustre: 128505:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3633.988596] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3640.954820] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 3641.951087] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 3641.959349] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 3642.091062] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3651.038104] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount (FULL|IDLE) failover.node=192.0.2.254@tcp0,osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3655.158853] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 3655.339514] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 3658.884303] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 3659.059987] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 3662.816312] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3662.817808] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3662.819672] LustreError: Skipped 1 previous similar message [ 3662.822391] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3662.868512] Lustre: Skipped 1 previous similar message [ 3666.748338] Lustre: server umount lustre-OST0000 complete [ 3673.056834] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3673.077961] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3673.083996] Lustre: Skipped 1 previous similar message [ 3673.102953] Lustre: Skipped 2 previous similar messages [ 3677.957405] Lustre: server umount lustre-MDT0000 complete [ 3682.317487] LustreError: 127342:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787376537 with bad export cookie 17159302697729957667 [ 3682.322404] LustreError: MGC192.168.202.147@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3682.328084] LustreError: 127342:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 3682.798258] Lustre: server umount lustre-MDT0001 complete [ 3691.033237] Lustre: DEBUG MARKER: == conf-sanity test 90a: check max_mod_rpcs_in_flight is enforced ========================================================== 01:29:04 (1787376544) [ 3697.567869] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3697.966905] LustreError: 130916:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3697.989313] LustreError: 130916:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 5 previous similar messages [ 3698.044101] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3703.267897] LustreError: 130916:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3703.299205] LustreError: 130916:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 3703.454410] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 3707.362432] LustreError: 130917:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3712.137629] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3717.565959] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 3725.390136] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3732.430386] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3738.761283] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3739.164633] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3739.170756] Lustre: Skipped 1 previous similar message [ 3745.513827] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 3753.315569] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3760.742290] Lustre: *** cfs_fail_loc=159, val=0*** [ 3777.169861] Lustre: lustre-MDT0000: Client 7ed2313a-3a31-4652-b752-8d2ad5a29d94 (at 192.168.202.47@tcp) reconnecting [ 3779.356350] Lustre: *** cfs_fail_loc=159, val=0*** [ 3779.368738] Lustre: Skipped 5 previous similar messages [ 3795.001303] Lustre: lustre-MDT0000: Client 7ed2313a-3a31-4652-b752-8d2ad5a29d94 (at 192.168.202.47@tcp) reconnecting [ 3813.858542] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3813.863217] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3813.886183] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3814.884152] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3818.996161] Lustre: server umount lustre-OST0000 complete [ 3825.127258] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3825.128432] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3825.142112] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3825.167746] Lustre: Skipped 2 previous similar messages [ 3825.172696] Lustre: Skipped 2 previous similar messages [ 3829.832828] Lustre: server umount lustre-MDT0000 complete [ 3830.240176] LustreError: 133570:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3830.250211] LustreError: 133570:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 3833.685426] LustreError: 130898:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787376688 with bad export cookie 17159302697729958542 [ 3833.691737] LustreError: MGC192.168.202.147@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3833.696038] LustreError: 130898:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 3834.075912] Lustre: server umount lustre-MDT0001 complete [ 3847.622825] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing unload_modules_local [ 3850.841452] Key type lgssc unregistered [ 3851.046729] LNet: 134699:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3851.051610] LNetError: 134699:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3851.061199] LNet: Removed LNI 192.168.202.147@tcp [ 3851.817412] Key type .llcrypt unregistered [ 3851.819118] Key type ._llcrypt unregistered [ 3883.512896] Key type ._llcrypt registered [ 3883.516157] Key type .llcrypt registered [ 3883.674739] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing load_modules_local [ 3885.724637] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3885.992767] alg: No test for adler32 (adler32-zlib) [ 3887.406886] Lustre: Lustre: Build Version: 2.17.57_47_gae0d133 [ 3887.914773] LNet: Added LNI 192.168.202.147@tcp [8/256/0/180] [ 3889.703135] Key type lgssc registered [ 3891.542954] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3900.986876] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3901.025393] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3902.726738] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3907.399627] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 3912.182404] LustreError: 136116:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3912.204197] LustreError: 136116:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 3916.035907] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3916.395190] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3922.164869] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 3930.627215] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3936.609322] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3944.820075] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3945.137240] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3947.186358] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:42 to 0x280000401:65) [ 3952.112464] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 3959.367840] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3971.532152] Lustre: *** cfs_fail_loc=159, val=0*** [ 3987.039719] Lustre: lustre-MDT0000: Client 578b6fa1-c2c5-45be-98e6-d57e0d4ccf90 (at 192.168.202.47@tcp) reconnecting [ 3989.547410] Lustre: *** cfs_fail_loc=159, val=0*** [ 4004.997121] Lustre: lustre-MDT0001: Client 578b6fa1-c2c5-45be-98e6-d57e0d4ccf90 (at 192.168.202.47@tcp) reconnecting [ 4006.988401] Lustre: *** cfs_fail_loc=159, val=0*** [ 4006.990682] Lustre: Skipped 3 previous similar messages [ 4023.366401] Lustre: lustre-MDT0001: Client 578b6fa1-c2c5-45be-98e6-d57e0d4ccf90 (at 192.168.202.47@tcp) reconnecting [ 4027.342445] Lustre: 139152:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4030.110626] Lustre: *** cfs_fail_loc=159, val=0*** [ 4030.113102] Lustre: Skipped 4 previous similar messages [ 4045.850420] Lustre: lustre-MDT0000: Client 14af6869-0bb5-4386-8b2c-c784ac5ffc25 (at 192.168.202.47@tcp) reconnecting [ 4047.542475] Lustre: *** cfs_fail_loc=159, val=0*** [ 4047.544156] Lustre: Skipped 14 previous similar messages [ 4062.825374] Lustre: lustre-MDT0000: Client 14af6869-0bb5-4386-8b2c-c784ac5ffc25 (at 192.168.202.47@tcp) reconnecting [ 4063.961760] Lustre: 139446:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4068.458277] Lustre: server umount lustre-OST0000 complete [ 4074.467834] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4074.471419] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4074.489014] Lustre: Skipped 1 previous similar message [ 4074.502746] Lustre: Skipped 1 previous similar message [ 4079.585512] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4079.597133] Lustre: Skipped 1 previous similar message [ 4082.745195] Lustre: server umount lustre-MDT0000 complete [ 4084.705054] LustreError: 136116:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4085.727345] Lustre: 135429:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787376924/real 1787376924] req@0000000016335dba x1874200354686976/t0(0) o13->lustre-OST0000-osc-MDT0001@0@lo:7/4 lens 224/368 e 0 to 1 dl 1787376940 ref 1 fl Rpc:XQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 4085.728271] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4085.746728] Lustre: 135429:0:(client.c:2503:ptlrpc_expire_one_request()) Skipped 2 previous similar messages [ 4085.786213] LustreError: 136117:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4085.815620] LustreError: 136117:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 4086.143905] LustreError: 139273:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787376941 with bad export cookie 1656867859266154015 [ 4086.148975] LustreError: MGC192.168.202.147@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4086.153111] LustreError: 139273:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4086.540324] Lustre: server umount lustre-MDT0001 complete [ 4097.489560] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing unload_modules_local [ 4100.713517] Key type lgssc unregistered [ 4100.940606] LNet: 140500:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4100.947888] LNetError: 140500:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4100.963604] LNet: Removed LNI 192.168.202.147@tcp [ 4101.840049] Key type .llcrypt unregistered [ 4101.842491] Key type ._llcrypt unregistered [ 4127.393557] Key type ._llcrypt registered [ 4127.396108] Key type .llcrypt registered [ 4127.515621] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing load_modules_local [ 4128.815127] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4128.991908] alg: No test for adler32 (adler32-zlib) [ 4130.289312] Lustre: Lustre: Build Version: 2.17.57_47_gae0d133 [ 4130.484982] LNet: Added LNI 192.168.202.147@tcp [8/256/0/180] [ 4132.135213] Key type lgssc registered [ 4133.443528] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4141.197969] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4141.251309] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4142.665496] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4147.720425] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 4152.806162] LustreError: 141917:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4152.820882] LustreError: 141917:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 4156.368483] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4156.888783] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4161.481461] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 4168.077619] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4173.638412] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4179.532149] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4179.852811] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4181.939242] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:97) [ 4185.617732] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 4188.095563] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:129) [ 4192.343671] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4198.224941] Lustre: 144500:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4200.963589] Lustre: 144549:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4202.253626] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 in log params [ 4203.188698] Lustre: Setting parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=8 in log params [ 4206.671657] Lustre: 144695:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre/mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 (mode = 0) failed: rc = -17 [ 4207.618226] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=7 in log params [ 4208.591569] Lustre: 144793:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4213.734078] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4213.741902] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4213.744639] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4213.754600] Lustre: Skipped 1 previous similar message [ 4213.773185] Lustre: Skipped 1 previous similar message [ 4218.849579] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4218.860103] Lustre: Skipped 1 previous similar message [ 4219.315740] Lustre: server umount lustre-OST0000 complete [ 4223.458117] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4223.461467] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4223.487436] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4229.088848] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4229.093889] Lustre: Skipped 2 previous similar messages [ 4229.946306] Lustre: server umount lustre-MDT0000 complete [ 4233.884105] LustreError: 141899:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787377088 with bad export cookie 7894560134504799261 [ 4233.903496] LustreError: 141899:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4233.907100] LustreError: MGC192.168.202.147@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4234.420940] Lustre: server umount lustre-MDT0001 complete [ 4246.016964] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing unload_modules_local [ 4249.501973] Key type lgssc unregistered [ 4249.827532] LNet: 145857:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4249.846619] LNetError: 145857:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4249.875050] LNet: Removed LNI 192.168.202.147@tcp [ 4251.039891] Key type .llcrypt unregistered [ 4251.043072] Key type ._llcrypt unregistered [ 4278.753370] Key type ._llcrypt registered [ 4278.756249] Key type .llcrypt registered [ 4278.856665] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing load_modules_local [ 4279.791723] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4279.814760] alg: No test for adler32 (adler32-zlib) [ 4281.037052] Lustre: Lustre: Build Version: 2.17.57_47_gae0d133 [ 4281.230346] LNet: Added LNI 192.168.202.147@tcp [8/256/0/180] [ 4282.839182] Key type lgssc registered [ 4284.274657] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4292.359794] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4292.408752] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4294.051743] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4298.789091] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 4303.338340] LustreError: 147281:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4303.354674] LustreError: 147281:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 4307.384738] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4307.769902] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4313.538951] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 4321.850468] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4327.645683] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4334.639724] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4335.212303] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4338.301081] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:161) [ 4342.456537] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 4350.273294] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4352.644030] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:129) [ 4357.583188] Lustre: *** cfs_fail_loc=159, val=0*** [ 4373.161473] Lustre: lustre-MDT0000: Client 1b5cb3f1-7246-4a96-af2f-8906c16ee23f (at 192.168.202.47@tcp) reconnecting [ 4393.954662] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4393.973169] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4393.985952] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4393.991127] Lustre: Skipped 1 previous similar message [ 4394.017244] Lustre: Skipped 1 previous similar message [ 4397.296517] Lustre: server umount lustre-OST0000 complete [ 4404.196395] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4404.199273] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4404.217338] Lustre: Skipped 1 previous similar message [ 4408.272966] Lustre: server umount lustre-MDT0000 complete [ 4409.314976] LustreError: 147281:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4409.329840] LustreError: 147281:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 4412.786314] LustreError: 149132:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787377267 with bad export cookie 11768231178079659129 [ 4412.793890] LustreError: MGC192.168.202.147@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4412.807550] LustreError: 149132:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4413.191870] Lustre: server umount lustre-MDT0001 complete [ 4426.086722] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing unload_modules_local [ 4429.229607] Key type lgssc unregistered [ 4429.526123] LNet: 150952:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4429.531252] LNetError: 150952:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4430.566917] LNet: Removed LNI 192.168.202.147@tcp [ 4431.807734] Key type .llcrypt unregistered [ 4431.821393] Key type ._llcrypt unregistered [ 4455.525490] Key type ._llcrypt registered [ 4455.529449] Key type .llcrypt registered [ 4455.645560] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing load_modules_local [ 4456.803736] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4456.916227] alg: No test for adler32 (adler32-zlib) [ 4457.976464] Lustre: Lustre: Build Version: 2.17.57_47_gae0d133 [ 4458.171415] LNet: Added LNI 192.168.202.147@tcp [8/256/0/180] [ 4459.815176] Key type lgssc registered [ 4461.114720] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4469.568606] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4469.587620] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4471.099752] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4476.312981] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 4480.492428] LustreError: 152369:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4480.512442] LustreError: 152369:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 4484.496077] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4485.080330] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4489.849960] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 4497.820552] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4504.027158] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4510.340729] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4510.703206] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4515.935664] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:170 to 0x280000401:193) [ 4518.338902] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 4526.545118] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4529.202734] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:161) [ 4552.571828] Lustre: lustre-MDT0001: evicting 5ba27399-411c-48e8-9e99-cf8948a04c44 (at 192.168.202.47@tcp) by administrative request [ 4552.582466] Lustre: Skipped 1 previous similar message [ 4582.372775] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4582.385701] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4582.394127] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4585.443181] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4585.443974] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4590.568104] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4590.577776] Lustre: Skipped 2 previous similar messages [ 4595.687655] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4595.704364] Lustre: Skipped 1 previous similar message [ 4596.191946] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 4596.547623] Lustre: server umount lustre-OST0000 complete [ 4602.849167] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4602.863668] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4602.896883] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4605.930516] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4607.234411] Lustre: server umount lustre-MDT0000 complete [ 4610.770699] LustreError: 152350:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787377465 with bad export cookie 13324536538353678545 [ 4610.771776] LustreError: MGC192.168.202.147@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4610.779543] LustreError: 152350:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4611.162751] Lustre: server umount lustre-MDT0001 complete [ 4622.590272] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing unload_modules_local [ 4625.727272] Key type lgssc unregistered [ 4625.989314] LNet: 156436:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4625.996864] LNetError: 156436:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4626.014947] LNet: Removed LNI 192.168.202.147@tcp [ 4626.902085] Key type .llcrypt unregistered [ 4626.903834] Key type ._llcrypt unregistered [ 4703.951116] Key type ._llcrypt registered [ 4703.954401] Key type .llcrypt registered [ 4704.096676] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing load_modules_local [ 4705.167531] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4705.219148] alg: No test for adler32 (adler32-zlib) [ 4706.233977] Lustre: Lustre: Build Version: 2.17.57_47_gae0d133 [ 4706.500907] LNet: Added LNI 192.168.202.147@tcp [8/256/0/180] [ 4708.183314] Key type lgssc registered [ 4709.327650] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4717.621949] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4717.672309] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4719.348159] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4724.519400] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 4729.825407] LustreError: 158927:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4729.859779] LustreError: 158927:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 4734.035283] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4734.486234] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4739.319511] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 4747.652974] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4754.050937] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4760.980689] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4761.309092] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4764.420979] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:195 to 0x280000401:225) [ 4767.552572] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 4776.141971] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4779.839584] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:193) [ 4790.250281] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4790.250526] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4790.276057] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4790.288160] Lustre: Skipped 1 previous similar message [ 4795.365756] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4795.381114] Lustre: Skipped 2 previous similar messages [ 4795.742114] Lustre: server umount lustre-OST0000 complete [ 4800.481987] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4800.485083] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4800.493201] Lustre: Skipped 1 previous similar message [ 4800.513834] Lustre: Skipped 1 previous similar message [ 4805.602186] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4805.612199] Lustre: Skipped 1 previous similar message [ 4810.728137] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4810.736591] Lustre: Skipped 1 previous similar message [ 4814.304070] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 4814.586683] Lustre: server umount lustre-MDT0000 complete [ 4815.840695] LustreError: 158927:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4815.878428] LustreError: 158927:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 4818.307191] LustreError: 158909:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787377673 with bad export cookie 7555235671935311974 [ 4818.318012] LustreError: 158909:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 4818.318212] LustreError: MGC192.168.202.147@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4818.731510] Lustre: server umount lustre-MDT0001 complete [ 4831.072634] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing unload_modules_local [ 4833.858354] Key type lgssc unregistered [ 4834.129735] LNet: 162495:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4834.135788] LNetError: 162495:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4834.148201] LNet: Removed LNI 192.168.202.147@tcp [ 4835.094196] Key type .llcrypt unregistered [ 4835.096986] Key type ._llcrypt unregistered [ 4861.933978] Key type ._llcrypt registered [ 4861.942056] Key type .llcrypt registered [ 4862.080406] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing load_modules_local [ 4863.882124] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4863.959673] alg: No test for adler32 (adler32-zlib) [ 4865.149104] Lustre: Lustre: Build Version: 2.17.57_47_gae0d133 [ 4865.484425] LNet: Added LNI 192.168.202.147@tcp [8/256/0/180] [ 4867.207230] Key type lgssc registered [ 4868.649949] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4885.472414] Lustre: DEBUG MARKER: == conf-sanity test 100: check lshowmount lists MGS, MDT, OST and 0@lo ========================================================== 01:48:58 (1787377738) [ 4901.226606] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_hostid [ 4912.171621] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing load_modules_local [ 4950.374414] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4950.409378] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4951.799670] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 4951.825149] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 4951.882119] Lustre: lustre-MDT0000: new disk, initializing [ 4951.934996] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4951.952488] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 4956.799502] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 4967.429747] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4967.578295] Lustre: 166762:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 4967.631467] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 4967.635228] Lustre: Skipped 1 previous similar message [ 4967.724488] Lustre: lustre-MDT0001: new disk, initializing [ 4967.876395] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4967.970000] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 4967.995102] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 4972.600490] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 4981.030913] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4986.697498] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4993.513831] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4993.690348] Lustre: lustre-OST0000: new disk, initializing [ 4993.693587] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 4993.698709] Lustre: 168730:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 4993.757500] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5000.590788] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 5002.384799] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 5002.413400] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 5002.682580] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 5012.122062] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 5023.202961] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5023.205564] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5023.230108] Lustre: Skipped 1 previous similar message [ 5023.249740] Lustre: Skipped 1 previous similar message [ 5028.327957] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5028.343326] Lustre: Skipped 1 previous similar message [ 5029.366858] Lustre: server umount lustre-OST0000 complete [ 5033.440900] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5033.444668] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5033.451871] Lustre: Skipped 1 previous similar message [ 5033.463386] Lustre: Skipped 1 previous similar message [ 5038.561519] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5038.572094] Lustre: Skipped 1 previous similar message [ 5039.806969] Lustre: server umount lustre-MDT0000 complete [ 5044.100729] LustreError: 166753:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787377899 with bad export cookie 17983823344453368395 [ 5044.104244] LustreError: MGC192.168.202.147@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5044.111445] LustreError: 166753:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 5044.536527] Lustre: server umount lustre-MDT0001 complete [ 5057.240476] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing unload_modules_local [ 5059.987411] Key type lgssc unregistered [ 5060.203563] LNet: 170649:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5060.207201] LNetError: 170649:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5060.225767] LNet: Removed LNI 192.168.202.147@tcp [ 5061.305806] Key type .llcrypt unregistered [ 5061.310122] Key type ._llcrypt unregistered [ 5087.616320] Key type ._llcrypt registered [ 5087.618845] Key type .llcrypt registered [ 5087.755929] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing load_modules_local [ 5088.981932] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5089.018788] alg: No test for adler32 (adler32-zlib) [ 5090.205881] Lustre: Lustre: Build Version: 2.17.57_47_gae0d133 [ 5090.463171] LNet: Added LNI 192.168.202.147@tcp [8/256/0/180] [ 5092.167170] Key type lgssc registered [ 5093.560835] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5102.956429] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5102.990165] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5104.418795] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5108.815827] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 5114.869838] LustreError: 172068:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5114.902448] LustreError: 172068:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 5117.389441] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5117.737267] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5122.533684] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 5130.304373] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 5137.283705] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 5144.806809] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5145.050398] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5149.205793] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:3 to 0x280000401:33) [ 5152.131311] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 5160.512611] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 5170.191104] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5170.219534] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5170.250182] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5170.258320] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5170.269031] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5171.274534] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5171.295861] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5171.307295] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5171.321429] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5171.337680] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5172.357390] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5172.372526] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5172.380736] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5172.383687] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5172.391924] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5173.346583] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5174.336983] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5175.297739] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5175.301595] Lustre: Skipped 2 previous similar messages [ 5175.316162] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5175.324139] Lustre: Skipped 2 previous similar messages [ 5175.328227] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5175.331068] Lustre: Skipped 2 previous similar messages [ 5175.336626] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5175.346808] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5175.352336] Lustre: Skipped 2 previous similar messages [ 5176.277613] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5177.040115] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5177.972454] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5178.929655] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5180.133722] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5180.140320] Lustre: Skipped 4 previous similar messages [ 5180.162984] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5180.176084] Lustre: Skipped 4 previous similar messages [ 5180.185611] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5180.191805] Lustre: Skipped 4 previous similar messages [ 5180.194641] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5180.204339] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5180.214433] Lustre: Skipped 4 previous similar messages [ 5181.043272] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5182.003669] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5182.772784] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5183.910973] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5184.864498] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5185.730360] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5186.676261] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5187.652036] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5188.347280] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5188.351823] Lustre: Skipped 8 previous similar messages [ 5188.369396] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5188.376618] Lustre: Skipped 8 previous similar messages [ 5188.380530] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5188.383031] Lustre: Skipped 8 previous similar messages [ 5188.387616] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5188.398926] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5188.413137] Lustre: Skipped 8 previous similar messages [ 5189.368214] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5190.458592] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5191.506327] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5192.405474] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5193.319142] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5194.155674] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5194.934426] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5195.867358] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5196.655473] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5197.489787] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5198.250105] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5198.894402] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5199.793961] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5200.479443] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5201.370922] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5202.054933] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5202.698508] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5203.473934] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5204.291782] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5205.221969] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5205.226381] Lustre: Skipped 19 previous similar messages [ 5205.238535] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5205.242848] Lustre: Skipped 19 previous similar messages [ 5205.244823] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5205.248557] Lustre: Skipped 19 previous similar messages [ 5205.250436] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5205.255525] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5205.273608] Lustre: Skipped 19 previous similar messages [ 5206.127637] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5206.994265] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5207.917380] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5208.921902] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5209.866888] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5210.785377] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5211.661843] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5212.419849] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5213.249667] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5214.085339] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5215.124392] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5215.875037] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5216.678321] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5217.558301] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5218.409875] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5219.328252] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5220.149421] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5221.008069] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5221.838877] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5222.440225] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5223.348764] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5224.147328] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5224.921555] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5225.941775] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5226.715632] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5227.660293] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5228.437204] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5229.574213] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5230.406238] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5231.329745] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5232.210957] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5233.034663] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5233.981246] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5235.080604] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5235.930783] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5237.048237] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5237.912678] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5237.917143] Lustre: Skipped 36 previous similar messages [ 5237.952690] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5237.967527] Lustre: Skipped 36 previous similar messages [ 5237.971767] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5237.977028] Lustre: Skipped 36 previous similar messages [ 5237.981576] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5238.001763] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5238.023694] Lustre: Skipped 36 previous similar messages [ 5239.197771] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5240.117747] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5241.388353] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5242.330472] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5243.417420] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5244.714118] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5245.472107] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5246.810828] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5247.861403] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5249.116309] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5250.216469] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5251.265928] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5252.063510] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5252.712879] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5253.696848] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5254.671218] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5255.715501] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5256.505333] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5257.419149] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5258.163357] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5259.014296] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5259.716749] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5260.668094] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5261.742046] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5262.717560] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5263.477842] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5264.562559] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5265.500915] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5266.400362] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5267.393491] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5268.278561] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5269.101152] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5270.172943] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5271.107865] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5272.179118] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5273.080753] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5274.208067] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5275.021612] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5275.916748] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5276.825934] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5277.998581] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5278.845622] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5279.808515] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5280.571708] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5281.489276] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5282.354031] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5283.140096] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5284.075985] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5285.024621] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5285.830197] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5286.935956] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5287.867838] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5288.782182] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5289.662557] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5290.623141] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5291.694915] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5292.522115] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5293.362552] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5294.231316] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5295.367910] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5296.356749] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5297.428142] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5298.377088] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5299.643368] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5300.681834] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5301.609890] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5302.537551] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5302.543048] Lustre: Skipped 66 previous similar messages [ 5302.556531] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5302.566261] Lustre: Skipped 66 previous similar messages [ 5302.571237] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5302.573796] Lustre: Skipped 66 previous similar messages [ 5302.585811] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5302.593703] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5302.602640] Lustre: Skipped 66 previous similar messages [ 5303.510765] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5304.498821] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5305.494385] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5306.433045] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5307.374401] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5308.200345] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5309.280507] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5309.976232] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5310.883342] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5311.786983] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5312.729607] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5313.869317] LustreError: 172087:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5313.883394] LustreError: 172087:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5313.905157] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5314.854395] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5315.895554] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5316.610776] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5317.494657] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5318.307155] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5319.344458] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5320.351896] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5321.207285] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5322.348957] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5323.383276] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5324.371303] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5325.370202] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5326.396203] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5327.277852] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5328.105492] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5329.006137] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5329.939952] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5330.924827] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5331.961707] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5332.964523] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5333.932614] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5334.989242] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5336.138904] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5337.088863] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5338.268860] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5339.210377] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5340.217335] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5341.132681] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5342.162870] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5342.980870] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5343.931818] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5344.773697] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5345.811859] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5346.677606] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5347.822344] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5348.852874] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5349.928574] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5350.870645] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5351.944082] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5352.907840] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5353.925566] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5354.847661] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5355.805582] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5356.774863] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5357.805643] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5358.823835] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5359.949905] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5360.783418] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5361.800735] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5362.694680] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5363.431899] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5364.261922] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5365.139958] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5366.049983] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5366.975706] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5368.083542] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5369.256796] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5370.022433] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5371.096508] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5372.070241] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5373.119377] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5374.182472] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5375.014202] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5376.031208] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5376.886923] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5377.610277] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5378.491082] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5379.299608] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5380.240695] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5381.030271] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5382.105838] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5382.997144] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5384.035063] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5384.996782] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5386.018199] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5387.015833] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5387.878055] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5388.785966] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5389.712246] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5390.604887] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5391.537525] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5392.395602] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5393.505821] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5394.269179] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5395.390263] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5396.299984] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5397.260836] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5398.264408] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5399.203703] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5400.130466] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5400.991623] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5402.042767] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5402.961053] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5403.946041] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5404.933142] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5405.913741] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5406.929192] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5407.715580] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5408.682220] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5409.534331] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5410.370728] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5411.197706] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5412.137280] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5413.145139] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5414.063739] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5414.907633] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5416.082578] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5417.021713] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5418.023894] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5418.924547] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5419.830086] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5420.723324] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5421.846624] LustreError: 172087:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5421.859729] LustreError: 172087:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5421.869545] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5422.741535] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5423.743950] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5424.703239] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5425.834178] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5426.700033] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5427.683991] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5428.591378] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5429.733959] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5430.884992] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5430.894023] Lustre: Skipped 133 previous similar messages [ 5430.916978] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5430.926571] Lustre: Skipped 133 previous similar messages [ 5430.944381] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5430.954328] Lustre: Skipped 133 previous similar messages [ 5430.967169] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5430.986864] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5430.996439] Lustre: Skipped 133 previous similar messages [ 5432.054979] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5433.107076] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5434.222665] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5435.312647] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5436.248104] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5437.232423] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5438.320694] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5439.380458] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5440.471921] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5441.360928] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5442.291053] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5443.167544] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5444.198420] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5445.257530] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5446.276986] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5447.498409] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5448.540865] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5449.594148] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5450.554539] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5451.841501] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5452.662548] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5453.696340] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5454.799324] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5455.772725] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5456.864789] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5457.678701] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5458.761231] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5459.957964] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5461.025576] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5462.061077] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5463.394497] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5464.439256] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5465.510934] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5466.616296] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5467.585438] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5468.676281] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5469.782805] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5470.518701] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5471.396084] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5472.274486] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5473.369734] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5474.636788] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5475.685406] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5476.720248] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5477.939799] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5479.093969] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5480.162168] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5481.445767] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5482.626728] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5483.513770] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5484.662802] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5485.791115] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5486.816584] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5487.636720] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5488.680492] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5489.798283] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5490.684441] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5491.746545] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5492.688204] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5493.711698] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5494.714757] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5495.602068] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5496.551720] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5497.536548] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5498.577966] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5499.526154] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5500.434561] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5501.515725] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5502.410802] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5503.318393] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5504.200048] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5505.413086] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5506.374319] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5507.488091] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5508.575527] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5509.497333] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5510.639043] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5511.553879] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5512.470850] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5513.417221] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5514.491550] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5515.383440] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5516.194416] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5517.002784] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5518.026709] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5518.996413] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5519.918626] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5520.850056] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5521.915604] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5522.814435] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5523.797343] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5524.575630] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5525.676082] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5526.629219] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5527.622341] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5528.594919] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5529.706070] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5530.812491] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5531.935115] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5532.857285] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5533.929898] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5534.922590] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5535.924303] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5536.969673] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5538.169528] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5539.221196] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5540.204854] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5541.317094] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5542.486453] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5543.502340] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5544.314330] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5545.362947] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5546.560380] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5547.427761] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5548.530844] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5549.596466] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5550.625339] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5551.334020] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5552.430833] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5553.638289] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5554.595790] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5555.992708] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5557.674766] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5559.005104] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5560.227604] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5561.547402] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5562.640397] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5563.795866] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5565.230254] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5566.224232] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5567.399515] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5568.809577] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5569.654816] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5571.286685] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5572.335308] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5573.054401] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5574.162542] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5575.195593] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5576.238353] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5577.302316] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5578.341181] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5579.451195] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5580.539882] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5581.555258] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5582.697277] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5583.760190] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5585.219553] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5586.143938] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5587.293800] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5588.361671] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5589.483204] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5590.478592] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5591.433642] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5592.590605] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5593.680756] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5594.659696] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5596.050623] LustreError: 172087:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5596.068352] LustreError: 172087:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5596.084393] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5597.287305] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5598.475197] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5599.576263] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5600.515616] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5601.639220] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5602.771404] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5603.844907] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5604.726916] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5605.908720] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5606.905544] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5608.147836] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5609.067411] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5610.455744] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5611.440527] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5612.486435] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5613.461823] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5614.685182] LustreError: 172087:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5614.695466] LustreError: 172087:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5614.704542] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5615.762499] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5616.456763] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5617.300471] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5618.311907] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5619.500270] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5620.309865] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5621.840161] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5622.957509] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5624.268437] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5625.301895] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5626.759033] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5628.055141] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5629.140673] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5630.416218] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5631.898601] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5632.965783] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5634.392918] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5635.929775] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5637.314517] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5638.983554] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5640.485802] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5641.812942] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5642.764498] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5643.848514] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5644.768392] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5645.853652] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5646.835279] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5647.743808] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5648.722875] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5649.881186] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5650.830192] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5651.841176] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5652.798194] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5653.900939] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5654.853665] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5656.056702] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5657.232933] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5658.196102] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5659.361703] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5660.178994] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5661.022996] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5661.985060] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5662.740983] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5663.619262] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5664.593775] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5665.544906] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5666.527775] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5667.698046] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5668.645934] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5669.548282] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5670.384881] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5671.465467] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5672.707474] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5673.636633] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5674.741044] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5675.553910] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5676.639206] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5677.744632] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5678.767666] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5679.653231] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5680.476125] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5681.537628] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5682.562790] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5683.527628] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5684.263698] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5685.205069] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5686.200094] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5687.088388] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5687.094317] Lustre: Skipped 241 previous similar messages [ 5687.113413] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5687.124357] Lustre: Skipped 241 previous similar messages [ 5687.131797] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5687.136035] Lustre: Skipped 241 previous similar messages [ 5687.140749] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5687.153871] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5687.173395] Lustre: Skipped 241 previous similar messages [ 5688.087381] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5688.983647] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5689.842081] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5691.019640] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5691.974417] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5693.101295] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5694.113565] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5695.402184] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5696.206797] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5697.466798] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5698.542115] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5699.611158] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5700.398892] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5701.388136] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5702.500061] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5703.570428] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5704.382352] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5705.422736] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5706.412562] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5707.365668] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5708.362289] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5709.343216] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5710.143657] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5711.154815] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5711.982519] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5712.841632] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5713.844276] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5714.993358] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5715.808971] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5716.786233] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5717.773964] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5719.061987] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5720.252931] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5721.264289] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5722.362824] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5723.278442] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5724.322854] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5725.123535] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5726.300388] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5727.607703] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5728.851558] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5729.984979] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5731.238247] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5732.236785] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5733.226349] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5734.253084] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5735.186808] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5736.141109] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5737.195073] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5738.072293] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5738.970300] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5739.882787] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5740.856659] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5741.687490] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5742.697638] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5743.616772] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5744.527214] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5745.697374] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5746.527946] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5747.614569] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5748.660355] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5749.547026] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5750.492077] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5751.382863] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5752.488995] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5753.388230] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5754.385375] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5755.408360] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5756.438671] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5757.502621] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5758.470858] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5759.419629] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5760.379873] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5761.380832] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5762.473145] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5763.384372] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5764.505800] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5765.723233] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5766.772369] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5767.652209] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5768.741170] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5769.710775] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5770.749970] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5771.846966] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5772.814064] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5773.823777] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5774.588064] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5775.697381] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5776.763814] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5777.865856] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5779.118512] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5779.989780] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5781.001757] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5781.864292] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5782.951943] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5783.794470] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5784.740658] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5785.686412] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5786.804967] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5787.748890] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5788.736113] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5789.660583] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5790.795527] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5791.752474] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5792.876801] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5793.939902] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5795.227506] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5796.339847] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5797.229900] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5798.352788] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5799.357763] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5800.460346] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5801.388696] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5802.371144] LustreError: 172087:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5802.381289] LustreError: 172087:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5802.392086] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5803.340057] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5804.514464] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5805.713159] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5806.763818] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5807.835184] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5809.188435] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5810.372540] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5811.453947] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5812.352869] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5813.381402] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5814.285749] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5815.476945] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5816.369955] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5817.342530] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5818.414699] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5819.239744] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5820.438579] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5821.363590] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5822.348236] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5823.424591] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5824.598793] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5825.640464] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5826.699128] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5827.960870] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5829.107482] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5830.224961] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5831.124839] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5831.927112] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5832.891873] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5834.084040] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5835.081912] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5836.522672] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5837.427480] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5838.782446] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5839.918732] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5840.858828] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5842.316508] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5843.366926] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5844.544785] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5845.804339] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5846.933166] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5847.895704] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5848.700663] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5849.433561] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5850.533813] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5851.417265] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5852.676329] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5853.632627] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5854.570731] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5855.739944] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5856.706029] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5857.754803] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5858.529903] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5859.497583] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5860.411497] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5861.417854] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5862.457784] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5863.608831] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5864.545543] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5865.628430] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5866.521383] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5867.600858] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5868.378195] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5869.321565] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5870.298714] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5871.501132] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5872.533569] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5873.320730] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5874.017813] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5874.952338] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5875.644326] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5876.450927] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5877.192283] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5878.084729] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5879.093394] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5880.111653] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5881.151975] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5881.980261] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5882.889307] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5883.840166] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5884.803579] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5885.833937] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5886.551211] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5887.580109] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5888.546842] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5889.574794] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5890.334605] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5891.239088] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5892.096310] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5893.192995] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5894.078326] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5894.844215] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5895.711453] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5896.564892] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5897.411759] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5898.419036] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5899.248383] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5900.085838] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5900.745530] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5901.688598] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5902.398484] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5903.211137] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5903.877766] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5904.797280] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5905.548940] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5906.338653] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5907.139304] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5907.923312] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5908.842431] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5909.686542] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5910.511616] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5911.264898] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5912.307591] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5913.324471] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5914.255330] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5915.321424] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5916.294671] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5917.131040] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5918.028333] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5918.790475] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5919.734828] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5920.556830] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5921.571190] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5922.456201] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5923.507958] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5924.407548] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5925.435964] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5926.312097] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5927.072378] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5927.890275] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5928.931328] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5929.906450] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5930.986339] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5931.786269] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5932.930281] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5934.046276] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5935.150134] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5936.226244] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5937.520070] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5938.780286] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5939.928611] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5941.095458] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5942.173570] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5943.453079] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5944.589872] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5945.886946] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5947.008407] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5948.384218] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5949.694936] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5951.038145] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5952.149187] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5953.237482] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5954.338754] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5955.338297] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5956.128655] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5957.169095] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5958.278170] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5959.246734] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5960.146907] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5961.129960] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5962.063334] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5962.883575] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5963.879417] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5964.807612] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5965.805806] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5966.573079] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5967.552697] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5968.553670] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5969.510438] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5970.552357] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5971.424251] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5972.368780] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5973.537745] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5974.490968] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5975.631772] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5976.606847] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5977.725087] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5978.637601] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5979.772395] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5980.920447] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5981.843376] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5982.796052] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5983.718727] LustreError: 172087:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5983.729388] LustreError: 172087:0:(osp_precreate.c:655:osp_precreate_send()) Skipped 1 previous similar message [ 5983.738861] LustreError: 172087:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5983.743027] LustreError: 172087:0:(osp_precreate.c:1359:osp_precreate_thread()) Skipped 1 previous similar message [ 5983.745118] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5984.581612] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5985.520692] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5986.483228] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5987.421670] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5988.274888] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5989.149537] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5990.147688] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5991.162335] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5992.070718] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5992.976378] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5994.002045] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5994.886765] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5995.787238] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5996.986574] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5997.989912] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5998.905598] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5999.634380] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6000.530685] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6001.428805] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6002.505946] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6003.478663] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6004.415588] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6005.521367] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6006.681532] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6007.587061] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6008.702768] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6009.656777] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6010.646474] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6011.757302] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6012.711663] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6013.635400] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6014.726013] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6015.612645] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6016.596423] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6017.402715] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6018.379656] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6019.272310] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6020.192622] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6020.938392] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6021.741869] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6022.629173] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6023.582159] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6024.526271] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6025.523749] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6026.566750] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6027.411682] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6028.313289] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6029.232617] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6030.312730] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6031.114208] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6031.838654] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6032.490391] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6033.456397] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6034.438665] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6035.505630] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6036.336669] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6037.369039] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6038.227584] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6039.080618] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6039.976634] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6040.821894] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6041.841197] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6042.968411] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6044.087560] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6045.026044] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6046.070566] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6047.064799] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6048.141839] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6049.115755] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6049.896356] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6050.666988] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6051.597784] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6052.461396] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6053.312699] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6054.040941] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6054.909093] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6056.053919] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6057.055036] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6057.986147] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6058.848817] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6059.749257] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6060.623090] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6061.572893] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6062.635948] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6063.590185] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6064.639570] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6065.692466] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6066.711547] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6067.748495] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6068.851342] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6069.841250] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6071.056207] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6071.991767] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6073.056031] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6073.952064] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6074.948849] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6075.598136] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6076.574196] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6077.423530] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6078.368907] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6079.261152] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6080.178957] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6081.245441] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6082.195606] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6083.189349] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6084.076061] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6085.055753] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6086.302389] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6087.271169] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6088.490520] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6089.600183] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6090.790164] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6091.737428] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6092.807212] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6094.009727] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6094.965496] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6096.078719] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6096.895665] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6097.754401] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6098.692611] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6099.592116] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6100.563641] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6101.710858] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6102.806748] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6103.903333] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6104.862987] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6106.054531] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6107.231972] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6108.162848] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6109.242221] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6110.289105] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6111.197792] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6112.216118] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6113.338484] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6114.193939] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6115.300315] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6116.285725] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6117.507260] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6118.523425] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6119.487217] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6120.366872] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6121.439865] LustreError: 172087:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 6121.449926] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6121.451361] LustreError: 172087:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 6122.513362] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6123.603913] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6124.604385] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6125.587081] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6126.453647] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6127.577587] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6128.632082] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6129.632593] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6130.875791] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6132.215763] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6133.352188] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6134.149514] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6135.212367] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6136.403167] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6137.482063] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6138.619164] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6139.547174] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6140.687129] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6141.625115] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6142.718491] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6143.688493] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6144.745124] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6145.813848] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6146.947861] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6148.106549] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6149.207095] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6150.302034] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6151.357980] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6152.644855] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6153.784555] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6154.705594] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6155.561498] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6156.408893] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6157.313937] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6158.082595] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6158.982303] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6159.990440] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6160.927067] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6161.771304] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6162.665803] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6163.593711] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6492.127982] LustreError: 173889:0:(ofd_dev.c:1780:ofd_create_hdl()) lustre-OST0000: unable to precreate: rc = -28 [ 6492.136068] LustreError: 172087:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -28 [ 6928.867621] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 6928.875925] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6928.900933] Lustre: Skipped 480 previous similar messages [ 6928.927419] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6929.891396] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6929.901494] Lustre: Skipped 1 previous similar message [ 6934.156490] Lustre: server umount lustre-OST0000 complete [ 6940.129615] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6940.142185] Lustre: Skipped 1 previous similar message [ 6944.205234] Lustre: server umount lustre-MDT0000 complete [ 6945.256128] LustreError: 172795:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6945.280374] LustreError: 172795:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 6948.314530] LustreError: 215236:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787379803 with bad export cookie 16448439544694129706 [ 6948.318611] LustreError: MGC192.168.202.147@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6948.324768] LustreError: 215236:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 6948.659920] Lustre: server umount lustre-MDT0001 complete [ 6959.359715] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing unload_modules_local [ 6962.264528] Key type lgssc unregistered [ 6962.598761] LNet: 227306:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 6962.606421] LNetError: 227306:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 6962.625465] LNet: Removed LNI 192.168.202.147@tcp [ 6963.382923] Key type .llcrypt unregistered [ 6963.384734] Key type ._llcrypt unregistered [ 6989.195978] Key type ._llcrypt registered [ 6989.197873] Key type .llcrypt registered [ 6989.325273] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing load_modules_local [ 6990.396158] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 6990.573193] alg: No test for adler32 (adler32-zlib) [ 6991.839528] Lustre: Lustre: Build Version: 2.17.57_47_gae0d133 [ 6992.166486] LNet: Added LNI 192.168.202.147@tcp [8/256/0/180] [ 6993.871173] Key type lgssc registered [ 6995.096525] Lustre: Echo OBD driver; http://www.lustre.org/ [ 7003.118049] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 7003.144759] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7004.508860] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7008.214400] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 7014.887489] LustreError: 228723:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7014.900508] LustreError: 228723:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 7017.240467] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7017.768862] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7022.536585] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 7030.953296] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 7037.465818] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 7046.676530] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7047.158482] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7049.282686] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49665 to 0x280000401:49696) [ 7055.462551] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 7064.493115] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 7070.502896] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 7072.650673] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7072.659135] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 7072.671371] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7072.701428] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 7072.706089] LustreError: 228743:0:(osp_precreate.c:1268:osp_precreate_thread()) cfs_fail_timeout id 2107 sleeping for 20000ms [ 7077.440788] Lustre: lustre-OST0000: Not available for connect from 192.168.202.47@tcp (stopping) [ 7077.865780] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 7077.881493] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7077.932992] Lustre: Skipped 1 previous similar message [ 7077.943247] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7077.945860] Lustre: Skipped 1 previous similar message [ 7081.316531] Lustre: server umount lustre-OST0000 complete [ 7092.784310] LustreError: 228743:0:(osp_precreate.c:1268:osp_precreate_thread()) cfs_fail_timeout id 2107 awake [ 7116.979521] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7117.272307] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7118.321056] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7118.333519] LustreError: lustre-OST0000-osc-MDT0001: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7118.351173] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 7118.364735] Lustre: Skipped 1 previous similar message [ 7125.201819] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 7135.929886] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 7138.275476] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid in FULL state after 0 sec [ 7146.934779] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff9b13a1d1c000.ost_server_uuid 50 [ 7148.470736] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff9b13a1d1c000.ost_server_uuid in IDLE state after 0 sec [ 7153.632388] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 7153.645720] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7153.667076] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7159.266543] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7159.270493] Lustre: Skipped 2 previous similar messages [ 7159.923360] Lustre: server umount lustre-OST0000 complete [ 7166.432932] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7166.435940] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7166.467637] Lustre: Skipped 1 previous similar message [ 7166.472376] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7170.799326] Lustre: server umount lustre-MDT0000 complete [ 7174.624891] LustreError: 228724:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7174.641280] LustreError: 228724:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 7175.274906] LustreError: 228705:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787380030 with bad export cookie 9774198011343308818 [ 7175.279758] LustreError: MGC192.168.202.147@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7175.290347] LustreError: 228705:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 7175.595932] Lustre: server umount lustre-MDT0001 complete [ 7188.726636] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing unload_modules_local [ 7192.403719] Key type lgssc unregistered [ 7192.813875] LNet: 233937:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 7192.832027] LNetError: 233937:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 7192.845940] LNet: Removed LNI 192.168.202.147@tcp [ 7193.751953] Key type .llcrypt unregistered [ 7193.756804] Key type ._llcrypt unregistered [ 7229.002956] Key type ._llcrypt registered [ 7229.007338] Key type .llcrypt registered [ 7229.140682] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing load_modules_local [ 7230.475342] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 7230.513824] alg: No test for adler32 (adler32-zlib) [ 7231.700236] Lustre: Lustre: Build Version: 2.17.57_47_gae0d133 [ 7231.964198] LNet: Added LNI 192.168.202.147@tcp [8/256/0/180] [ 7233.639165] Key type lgssc registered [ 7235.082805] Lustre: Echo OBD driver; http://www.lustre.org/ [ 7246.060505] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 7246.085171] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7247.666809] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7253.581651] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 7258.104893] LustreError: 235682:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7258.121973] LustreError: 235682:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 7263.214142] LustreError: 235683:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7263.442255] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7263.964716] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7268.836822] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 7272.174461] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 7279.445351] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7279.986191] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7286.164774] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 7293.321235] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49728) [ 7295.803565] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7296.020819] Lustre: lustre-OST0001: new disk, initializing [ 7296.027231] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 7296.033844] Lustre: 237990:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 7296.159252] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 7302.597496] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 7305.370380] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 7305.384399] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 7305.525174] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 7313.873195] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7321.117604] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 7335.905518] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7335.915732] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7335.928443] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7336.419299] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7336.439643] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7336.442690] Lustre: Skipped 2 previous similar messages [ 7336.461737] Lustre: Skipped 3 previous similar messages [ 7341.541510] LustreError: 235679:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7341.558801] Lustre: server umount lustre-MDT0000 complete [ 7341.559934] LustreError: 235679:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [ 7345.795716] LustreError: 235662:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787380200 with bad export cookie 16070538155919652303 [ 7345.810747] LustreError: MGC192.168.202.147@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7345.817946] LustreError: 235662:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 7346.116882] Lustre: server umount lustre-MDT0001 complete [ 7360.734985] Lustre: server umount lustre-OST0000 complete [ 7363.041122] Lustre: 234944:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787380201/real 1787380201] req@ffff9ffdabcbdf80 x1874203861149056/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1787380217 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 7363.096394] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7366.155989] Lustre: server umount lustre-OST0001 complete [ 7391.329765] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing load_modules_local [ 7403.486334] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7404.074228] LustreError: 241096:0:(ldlm_lib.c:1190:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7404.176523] Lustre: mylustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7408.821314] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 7409.648219] LustreError: 241097:0:(ldlm_lib.c:1190:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7419.315200] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7419.925140] Lustre: mylustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7426.135967] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 7429.786615] Lustre: 242240:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 7437.897950] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7438.344420] Lustre: mylustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7444.525334] LustreError: 242596:0:(ldlm_lib.c:1190:target_handle_connect()) mylustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7444.545547] LustreError: 242596:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 7445.210134] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 7450.618317] Lustre: mylustre-OST0000: new connection from mylustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49731 to 0x280000401:49760) [ 7454.758971] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7461.930451] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 7469.108212] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7472.943226] Lustre: 244110:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 7485.941286] Lustre: mylustre-MDT0000-lwp-OST0001: Connection to mylustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7485.945589] Lustre: mylustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7485.954720] Lustre: Skipped 2 previous similar messages [ 7485.956698] Lustre: Skipped 1 previous similar message [ 7491.042713] Lustre: mylustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7491.057535] Lustre: Skipped 3 previous similar messages [ 7496.166250] Lustre: mylustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7496.183303] Lustre: Skipped 3 previous similar messages [ 7497.183162] Lustre: mylustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 7497.606365] Lustre: server umount mylustre-MDT0000 complete [ 7501.282992] LustreError: 241092:0:(ldlm_lib.c:1190:target_handle_connect()) mylustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7501.302570] LustreError: 241092:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 6 previous similar messages [ 7501.500936] LustreError: 244312:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787380356 with bad export cookie 16070538155919655110 [ 7501.510950] LustreError: MGC192.168.202.147@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7501.718315] Lustre: server umount mylustre-MDT0001 complete [ 7515.942541] Lustre: server umount mylustre-OST0000 complete [ 7531.368540] Lustre: server umount mylustre-OST0001 complete [ 7553.408501] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing load_modules_local [ 7564.600108] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7565.110817] LustreError: 246427:0:(ldlm_lib.c:1190:target_handle_connect()) tfs-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7565.202227] Lustre: tfs-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7565.209324] Lustre: Skipped 1 previous similar message [ 7570.530955] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 7580.698141] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7588.443420] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 7591.774545] Lustre: 247569:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 7599.614061] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7606.787656] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 7610.295668] Lustre: tfs-OST0000: new connection from tfs-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49731 to 0x280000401:49792) [ 7615.488913] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7621.142273] Lustre: tfs-OST0001: new connection from tfs-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:35 to 0x2c0000401:65) [ 7622.952991] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 7630.058505] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7633.733093] Lustre: 249442:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 7642.594985] LustreError: tfs-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7642.603219] Lustre: tfs-MDT0000-osp-MDT0001: Connection to tfs-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7642.614279] Lustre: Skipped 1 previous similar message [ 7642.626314] Lustre: tfs-MDT0000: Not available for connect from 0@lo (stopping) [ 7642.633420] Lustre: Skipped 1 previous similar message [ 7644.284527] Lustre: server umount tfs-MDT0000 complete [ 7646.697217] LustreError: 246428:0:(ldlm_lib.c:1190:target_handle_connect()) tfs-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7646.714366] LustreError: 246428:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 11 previous similar messages [ 7648.438567] LustreError: 246402:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787380503 with bad export cookie 16070538155919656776 [ 7648.453105] LustreError: 246402:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 5 previous similar messages [ 7648.463178] LustreError: MGC192.168.202.147@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7649.006695] Lustre: server umount tfs-MDT0001 complete [ 7663.758500] Lustre: server umount tfs-OST0000 complete [ 7667.725782] Lustre: 234946:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787380506/real 1787380506] req@ffff9ffca3cb1500 x1874203861263872/t0(0) o400->tfs-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1787380522 ref 1 fl Rpc:EXNQr/200/ffffffff rc -5/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 7678.386836] Lustre: server umount tfs-OST0001 complete [ 7701.248348] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing load_modules_local [ 7712.682881] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7713.386492] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7713.396542] Lustre: Skipped 3 previous similar messages [ 7718.740419] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 7728.040860] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7734.432171] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 7737.729201] Lustre: 252836:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 7745.155304] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7752.977853] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 7759.948666] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49731 to 0x280000401:49824) [ 7761.822989] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7767.539229] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:35 to 0x2c0000401:97) [ 7768.602633] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 7776.364796] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7779.828871] Lustre: 254704:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 7786.760210] Lustre: DEBUG MARKER: == conf-sanity test 104a: Make sure user defined options are reflected in mount ========================================================== 02:37:20 (1787380640) [ 7805.409038] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7805.426559] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7805.450734] Lustre: Skipped 3 previous similar messages [ 7805.454624] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7809.838159] Lustre: server umount lustre-MDT0000 complete [ 7813.612112] LustreError: 255074:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7813.630541] LustreError: 255074:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 10 previous similar messages [ 7813.771085] LustreError: 251670:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787380668 with bad export cookie 16070538155919658442 [ 7813.781152] LustreError: MGC192.168.202.147@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7813.788821] LustreError: 251670:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 7820.225953] Lustre: server umount lustre-MDT0001 complete [ 7830.877825] Lustre: server umount lustre-OST0000 complete [ 7840.872282] Lustre: server umount lustre-OST0001 complete [ 7849.485468] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_hostid [ 7862.466438] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing load_modules_local [ 7905.590582] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7906.029901] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 7906.070309] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 7906.166751] Lustre: lustre-MDT0000: new disk, initializing [ 7906.281378] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 188-180 [ 7906.287054] Lustre: Skipped 3 previous similar messages [ 7906.310917] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 7911.166157] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 7923.524612] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7923.741722] Lustre: 257892:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 7923.854329] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 7923.859661] Lustre: Skipped 1 previous similar message [ 7924.023507] Lustre: lustre-MDT0001: new disk, initializing [ 7924.338568] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 7924.357286] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 7931.654859] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 7943.809882] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7944.370570] Lustre: lustre-OST0000: new disk, initializing [ 7944.375824] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 7944.390449] Lustre: 259473:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 7945.816360] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 7945.823331] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 7945.922092] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 7951.910924] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 7962.139527] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7962.502318] Lustre: lustre-OST0001: new disk, initializing [ 7962.504770] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 7962.542502] Lustre: 260346:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 7964.086711] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 7964.098533] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 7964.266234] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 7971.290342] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 7986.019819] Lustre: DEBUG MARKER: == conf-sanity test 104b: Mount uses last flock argument ========================================================== 02:40:39 (1787380839) [ 8005.950669] Lustre: DEBUG MARKER: == conf-sanity test 105: check file creation for ro and rw bind mnt pt ========================================================== 02:40:59 (1787380859) [ 8023.015548] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 8023.024648] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8023.025745] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8023.043368] Lustre: Skipped 6 previous similar messages [ 8023.069799] Lustre: Skipped 7 previous similar messages [ 8027.830765] Lustre: server umount lustre-OST0000 complete [ 8039.234670] Lustre: server umount lustre-MDT0000 complete [ 8044.106878] LustreError: 259484:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787380899 with bad export cookie 16070538155919659996 [ 8044.107740] LustreError: MGC192.168.202.147@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8044.115161] LustreError: 259484:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 8050.718989] Lustre: server umount lustre-MDT0001 complete [ 8065.512313] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing unload_modules_local [ 8139.234149] Lustre: server umount lustre-OST0001 complete [ 8147.126929] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_hostid [ 8165.097541] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing load_modules_local [ 8203.334769] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8203.664184] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 8203.700063] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 8203.770268] Lustre: lustre-MDT0000: new disk, initializing [ 8203.881238] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8203.890291] Lustre: Skipped 3 previous similar messages [ 8203.903547] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 8209.437759] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 8221.843385] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8222.027945] Lustre: 265199:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 8222.092797] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 8222.098277] Lustre: Skipped 1 previous similar message [ 8222.229576] Lustre: lustre-MDT0001: new disk, initializing [ 8222.481513] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 8222.494547] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 8227.711882] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 8239.502335] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 8247.019273] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 8254.622957] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8254.993521] Lustre: lustre-OST0000: new disk, initializing [ 8254.997357] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 8255.011934] Lustre: 267168:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8257.007438] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 8257.019838] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 8257.206357] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 8261.000853] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 8271.120580] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8282.593071] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 8282.601067] LustreError: Skipped 1 previous similar message [ 8282.606981] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8282.628133] Lustre: Skipped 4 previous similar messages [ 8282.635411] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8282.642868] Lustre: Skipped 7 previous similar messages [ 8284.422047] Lustre: server umount lustre-OST0000 complete [ 8294.442192] Lustre: server umount lustre-MDT0000 complete [ 8298.469019] LustreError: 265207:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8298.487021] LustreError: 265207:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 12 previous similar messages [ 8298.598747] LustreError: 266004:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787381153 with bad export cookie 16070538155919661466 [ 8298.612143] LustreError: MGC192.168.202.147@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8298.931493] Lustre: server umount lustre-MDT0001 complete [ 8311.234783] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing unload_modules_local [ 8314.237710] Key type lgssc unregistered [ 8314.553396] LNet: 268933:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8314.564952] LNetError: 268933:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8314.592323] LNet: Removed LNI 192.168.202.147@tcp [ 8315.821145] Key type .llcrypt unregistered [ 8315.822722] Key type ._llcrypt unregistered [ 8345.962531] Key type ._llcrypt registered [ 8345.965639] Key type .llcrypt registered [ 8346.096579] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing load_modules_local [ 8347.536860] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8347.776207] alg: No test for adler32 (adler32-zlib) [ 8348.914112] Lustre: Lustre: Build Version: 2.17.57_47_gae0d133 [ 8349.135964] LNet: Added LNI 192.168.202.147@tcp [8/256/0/180] [ 8350.815572] Key type lgssc registered [ 8352.988796] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8363.424629] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 8363.439773] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8364.930531] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8370.785871] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 8374.243893] LustreError: 270397:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8374.278803] LustreError: 270397:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 8379.374112] LustreError: 270398:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8380.353249] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8380.919613] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8386.206485] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 8395.369254] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 8402.181388] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 8410.607839] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8410.892588] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8418.773092] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 8427.712737] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8428.996407] Lustre: Setting parameter lustre-OST0000.ost.unknown_param=50 in log lustre-OST0000 [ 8434.469462] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8434.472764] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8437.809906] Lustre: server umount lustre-OST0000 complete [ 8442.339629] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8442.346946] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8442.366308] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8442.371168] Lustre: Skipped 1 previous similar message [ 8444.898732] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8444.900560] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8444.930720] Lustre: Skipped 1 previous similar message [ 8448.672534] Lustre: server umount lustre-MDT0000 complete [ 8450.022649] LustreError: 270397:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8450.044498] LustreError: 270397:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 8452.710442] LustreError: 270378:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787381307 with bad export cookie 8422169270334862270 [ 8452.714376] LustreError: MGC192.168.202.147@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8452.722871] LustreError: 270378:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 8453.382823] Lustre: server umount lustre-MDT0001 complete [ 8466.622328] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing unload_modules_local [ 8469.923239] Key type lgssc unregistered [ 8470.234157] LNet: 273985:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8470.251913] LNetError: 273985:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8470.280677] LNet: Removed LNI 192.168.202.147@tcp [ 8471.578116] Key type .llcrypt unregistered [ 8471.581139] Key type ._llcrypt unregistered [ 8492.744714] Key type ._llcrypt registered [ 8492.753225] Key type .llcrypt registered [ 8492.880734] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing load_modules_local [ 8494.453983] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8494.569182] alg: No test for adler32 (adler32-zlib) [ 8495.789958] Lustre: Lustre: Build Version: 2.17.57_47_gae0d133 [ 8496.213965] LNet: Added LNI 192.168.202.147@tcp [8/256/0/180] [ 8498.025200] Key type lgssc registered [ 8499.634938] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8510.125678] Lustre: lustre-OST0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 8510.136633] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8525.599489] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 8530.719328] LustreError: 275182:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.202.147@tcp: failed processing log, type 4: rc = -110 [ 8557.535178] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8564.769459] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 8573.712709] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8576.433047] Lustre: server umount lustre-OST0000 complete [ 8595.788666] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing unload_modules_local [ 8600.120214] Key type lgssc unregistered [ 8600.525421] LNet: 276818:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8600.542795] LNetError: 276818:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8600.570079] LNet: Removed LNI 192.168.202.147@tcp [ 8601.828356] Key type .llcrypt unregistered [ 8601.833906] Key type ._llcrypt unregistered [ 8648.833870] Key type ._llcrypt registered [ 8648.838351] Key type .llcrypt registered [ 8648.988169] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing load_modules_local [ 8650.356075] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8650.374819] alg: No test for adler32 (adler32-zlib) [ 8651.498325] Lustre: Lustre: Build Version: 2.17.57_47_gae0d133 [ 8651.706136] LNet: Added LNI 192.168.202.147@tcp [8/256/0/180] [ 8653.367206] Key type lgssc registered [ 8654.796176] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8671.165943] loop0: detected capacity change from 0 to 204800000 [ 8671.412694] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8671.442389] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8671.759767] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8673.174348] loop0: detected capacity change from 0 to 204800000 [ 8673.455730] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8673.475677] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8673.968657] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8675.354138] loop0: detected capacity change from 0 to 204800000 [ 8675.610812] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8675.631611] blk_update_request: operation not supported error, dev loop0, sector 8224 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8675.835823] blk_update_request: operation not supported error, dev loop0, sector 58264 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8677.518976] loop0: detected capacity change from 0 to 204800000 [ 8677.817955] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8677.829964] blk_update_request: operation not supported error, dev loop0, sector 8224 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8678.032149] blk_update_request: operation not supported error, dev loop0, sector 58264 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8679.083229] loop0: detected capacity change from 0 to 204800000 [ 8679.137255] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 8684.184426] loop0: detected capacity change from 0 to 204800000 [ 8684.211366] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 8689.038160] loop0: detected capacity change from 0 to 204800000 [ 8689.052290] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 8693.115167] loop0: detected capacity change from 0 to 204800000 [ 8693.137234] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 8697.495254] loop0: detected capacity change from 0 to 204800000 [ 8697.594302] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 8697.644995] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8697.771328] Lustre: 280232:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 8697.778495] Lustre: 280232:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000003:0xb:0x0]: rc = 0 [ 8697.796465] Lustre: 280232:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000-MDT0000' with [0x200000005:0x1:0x0]: rc = 0 [ 8697.806257] Lustre: 280232:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000-MDT0000' with [0x200000005:0x2:0x0]: rc = 0 [ 8697.819480] Lustre: 280232:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000003:0xc:0x0]: rc = 0 [ 8697.829600] Lustre: 280232:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000003:0xa:0x0]: rc = 0 [ 8697.840438] Lustre: 280232:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000-MDT0000' with [0x200000005:0x3:0x0]: rc = 0 [ 8697.854950] Lustre: 280232:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index 'nodemap' with [0x200000003:0x3:0x0]: rc = 0 [ 8697.873574] Lustre: 280232:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2020000' with [0x200000006:0x2020000:0x0]: rc = 0 [ 8697.897483] Lustre: 280232:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x20000' with [0x200000006:0x20000:0x0]: rc = 0 [ 8697.923493] Lustre: 280232:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1020000' with [0x200000006:0x1020000:0x0]: rc = 0 [ 8697.949753] Lustre: 280232:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000006:0x1010000:0x0]: rc = 0 [ 8697.963283] Lustre: 280232:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000006:0x2010000:0x0]: rc = 0 [ 8697.986554] Lustre: 280232:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000006:0x10000:0x0]: rc = 0 [ 8704.488324] Lustre: server umount MGS complete [ 8705.598851] loop0: detected capacity change from 0 to 204800000 [ 8705.738107] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8706.150150] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8706.208973] Lustre: lustre-MDT0000: Aborting client recovery [ 8707.305103] loop1: detected capacity change from 0 to 204800000 [ 8707.448150] LDISKFS-fs (loop1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8707.544256] Lustre: 280665:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 8707.565937] Lustre: 280665:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x2010000-MDT0001' with [0x200000005:0x3:0x0]: rc = 0 [ 8707.582944] Lustre: 280665:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x1010000-MDT0001' with [0x200000005:0x2:0x0]: rc = 0 [ 8707.595514] Lustre: 280665:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x1010000' with [0x200000003:0x5:0x0]: rc = 0 [ 8707.618868] Lustre: 280665:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x2010000' with [0x200000003:0x6:0x0]: rc = 0 [ 8707.647075] Lustre: 280665:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x10000' with [0x200000003:0x4:0x0]: rc = 0 [ 8707.661720] Lustre: 280665:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x10000-MDT0001' with [0x200000005:0x1:0x0]: rc = 0 [ 8707.698536] Lustre: Found index 1 for lustre-MDT0001, updating log [ 8707.870458] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8707.968278] Lustre: lustre-MDT0001: Aborting client recovery [ 8709.052446] loop2: detected capacity change from 0 to 204800000 [ 8709.270754] LDISKFS-fs (loop2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8709.340175] Lustre: 280744:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 8709.357986] Lustre: 280744:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 8709.375386] Lustre: 280744:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x2020000-OST0000' with [0x200000005:0x3:0x0]: rc = 0 [ 8709.386343] Lustre: 280744:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 8709.398761] Lustre: 280744:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x20000-OST0000' with [0x200000005:0x1:0x0]: rc = 0 [ 8709.414437] Lustre: 280744:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 8709.426808] Lustre: 280744:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x1020000-OST0000' with [0x200000005:0x2:0x0]: rc = 0 [ 8709.538401] Lustre: Found index 0 for lustre-OST0000, updating log [ 8709.570738] Lustre: 280762:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8709.642976] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8709.661408] Lustre: lustre-OST0000: Aborting recovery [ 8710.710601] loop3: detected capacity change from 0 to 204800000 [ 8710.990073] LDISKFS-fs (loop3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8711.115199] Lustre: 280827:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 8711.135058] Lustre: 280827:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 8711.146890] Lustre: 280827:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x2020000-OST0001' with [0x200000005:0x3:0x0]: rc = 0 [ 8711.157289] Lustre: 280827:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 8711.165962] Lustre: 280827:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x1020000-OST0001' with [0x200000005:0x2:0x0]: rc = 0 [ 8711.175352] Lustre: 280827:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x20000-OST0001' with [0x200000005:0x1:0x0]: rc = 0 [ 8711.183399] Lustre: 280827:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 8711.196495] Lustre: Found index 1 for lustre-OST0001, updating log [ 8711.226128] Lustre: 280837:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8711.303369] Lustre: lustre-OST0001: Aborting recovery [ 8716.401830] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 8716.421701] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 8716.424380] LustreError: 280609:0:(lod_dev.c:511:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 10, retries 0, failed: rc = -108 [ 8716.432958] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000bd0 [ 8716.472687] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000bd0 [ 8717.418233] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:5 to 0x280000400:33) [ 8717.430596] Lustre: lustre-OST0001: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:5 to 0x2c0000400:33) [ 8720.527971] LustreError: 280575:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0000: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0000 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0000/identity_upcall, time 673us: rc = -2 [ 8721.032292] LustreError: 280574:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0000: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0000 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0000/identity_upcall, time 566us: rc = -2 [ 8721.064892] LustreError: 280574:0:(mdt_identity.c:115:mdt_identity_do_upcall()) Skipped 18 previous similar messages [ 8738.280376] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8738.302436] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8738.311648] Lustre: Skipped 3 previous similar messages [ 8740.173386] Lustre: server umount lustre-MDT0000 complete [ 8741.199532] LustreError: 280558:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787381596 with bad export cookie 1000000262640036772 [ 8741.206652] LustreError: MGC192.168.202.147@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8741.214817] LustreError: 280558:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 8743.391750] LustreError: 280574:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8743.391988] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8743.398134] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 8743.415606] LustreError: 280574:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [ 8743.444592] Lustre: Skipped 4 previous similar messages [ 8746.403558] LustreError: 280573:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8746.404402] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 8746.447376] Lustre: Skipped 2 previous similar messages [ 8748.008392] Lustre: server umount lustre-MDT0001 complete [ 8755.458596] Lustre: server umount lustre-OST0000 complete [ 8762.931609] Lustre: server umount lustre-OST0001 complete [ 8771.872395] Lustre: DEBUG MARKER: == conf-sanity test 109a: test lctl clear_conf fsname ==== 02:53:45 (1787381625) [ 8802.893344] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_hostid [ 8812.229496] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing load_modules_local [ 8847.785286] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8848.088871] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 8848.167930] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 8848.395608] Lustre: lustre-MDT0000: new disk, initializing [ 8848.547753] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8848.553593] Lustre: Skipped 1 previous similar message [ 8848.576230] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 8855.457563] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 8870.125148] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8870.231393] Lustre: 284906:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 8870.312374] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 8870.315142] Lustre: Skipped 1 previous similar message [ 8870.383341] Lustre: lustre-MDT0001: new disk, initializing [ 8870.522784] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8870.559513] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 8870.575396] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 8875.429336] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 8885.374873] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 8891.050315] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 8900.096311] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8900.422213] Lustre: lustre-OST0000: new disk, initializing [ 8900.432243] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 8900.445313] Lustre: 286877:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8900.604612] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8907.420973] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 8907.430897] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 8907.535706] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 8907.583495] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 8915.775516] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8921.141976] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [ 8930.033858] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [ 8937.239899] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [ 8984.549203] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 8984.560982] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8984.562648] LustreError: Skipped 1 previous similar message [ 8984.570887] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8984.582825] Lustre: Skipped 1 previous similar message [ 8984.605372] Lustre: Skipped 1 previous similar message [ 8986.633303] Lustre: server umount lustre-OST0000 complete [ 8993.248273] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8993.264377] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8993.286477] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8996.615283] Lustre: server umount lustre-MDT0000 complete [ 8999.905117] LustreError: 284912:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8999.925182] LustreError: 284912:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 9001.390732] LustreError: 287733:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787381856 with bad export cookie 1000000262640039593 [ 9001.398599] LustreError: MGC192.168.202.147@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9001.408194] LustreError: 287733:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 9001.775160] Lustre: server umount lustre-MDT0001 complete [ 9011.855343] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9017.069165] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 9022.384468] Lustre: server umount MGS complete [ 9033.784899] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9034.198948] LustreError: 290739:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9034.233494] LustreError: 290752:0:(qmt_pool.c:1639:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [ 9034.290921] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9039.240871] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 9039.338737] LustreError: 290740:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9048.562988] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9053.813067] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 9060.938372] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 9066.207946] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 9072.767987] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9073.206783] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9073.210925] Lustre: Skipped 1 previous similar message [ 9079.104639] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 9086.528610] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9120.231914] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 9120.239241] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9120.243427] LustreError: Skipped 1 previous similar message [ 9120.258634] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9120.260174] Lustre: Skipped 2 previous similar messages [ 9120.269790] Lustre: Skipped 3 previous similar messages [ 9123.927206] Lustre: server umount lustre-OST0000 complete [ 9130.467205] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9130.467205] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9133.966959] Lustre: server umount lustre-MDT0000 complete [ 9135.585725] LustreError: 291468:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9135.609543] LustreError: 291468:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 9138.007530] LustreError: 290720:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787381992 with bad export cookie 1000000262640041420 [ 9138.010163] LustreError: MGC192.168.202.147@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9138.018780] LustreError: 290720:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 9138.438434] Lustre: server umount lustre-MDT0001 complete [ 9151.634341] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing unload_modules_local [ 9154.281622] Key type lgssc unregistered [ 9154.557263] LNet: 294691:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 9154.568337] LNetError: 294691:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 9154.609050] LNet: Removed LNI 192.168.202.147@tcp [ 9155.620189] Key type .llcrypt unregistered [ 9155.623093] Key type ._llcrypt unregistered [ 9191.944436] Key type ._llcrypt registered [ 9191.948486] Key type .llcrypt registered [ 9192.108292] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_hostid [ 9207.351130] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing load_modules_local [ 9208.521258] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 9208.653758] alg: No test for adler32 (adler32-zlib) [ 9209.664855] Lustre: Lustre: Build Version: 2.17.57_47_gae0d133 [ 9209.944384] LNet: Added LNI 192.168.202.147@tcp [8/256/0/180] [ 9211.615679] Key type lgssc registered [ 9212.744559] Lustre: Echo OBD driver; http://www.lustre.org/ [ 9244.917273] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 9244.939413] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9246.169874] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 9246.227508] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 9246.422333] Lustre: lustre-MDT0000: new disk, initializing [ 9246.554972] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9246.575205] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 9251.243623] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 9261.893517] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9262.065800] Lustre: 298664:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 9262.135375] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 9262.144340] Lustre: Skipped 1 previous similar message [ 9262.310756] Lustre: lustre-MDT0001: new disk, initializing [ 9262.452157] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9262.504706] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 9262.521896] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 9267.326619] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 9276.791926] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 9282.373858] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 9289.256345] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9289.554597] Lustre: lustre-OST0000: new disk, initializing [ 9289.560128] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 9289.564920] Lustre: 300631:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9289.644975] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9294.931545] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 9294.949067] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 9295.138127] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 9296.289751] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 9306.730665] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9314.458256] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [ 9327.587568] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [ 9335.597348] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [ 9387.489363] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 9387.504056] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9387.525119] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9387.529947] Lustre: Skipped 1 previous similar message [ 9392.580795] Lustre: server umount lustre-OST0000 complete [ 9397.728297] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9397.729832] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9397.744240] Lustre: Skipped 2 previous similar messages [ 9397.756949] Lustre: Skipped 1 previous similar message [ 9402.848218] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9402.855945] Lustre: Skipped 1 previous similar message [ 9403.879852] Lustre: server umount lustre-MDT0000 complete [ 9407.820226] LustreError: 298654:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787382262 with bad export cookie 5607381880926129825 [ 9407.824909] LustreError: MGC192.168.202.147@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9407.844912] LustreError: 298654:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 9408.103609] Lustre: server umount lustre-MDT0001 complete [ 9417.256074] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9422.863326] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 9428.034481] Lustre: server umount MGS complete [ 9440.222730] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9440.858201] LustreError: 304605:0:(qmt_pool.c:1639:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [ 9440.941699] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9445.937419] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 9446.376348] LustreError: 304592:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9446.407217] LustreError: 304592:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 9450.478690] LustreError: 304593:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9455.433352] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9455.678067] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9460.450804] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 9467.446789] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 9473.340283] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 9480.061546] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9480.171833] LustreError: 306412:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9480.332671] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9486.097781] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 9492.369578] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9521.637804] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 9521.641887] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9521.654162] LustreError: Skipped 1 previous similar message [ 9521.660374] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9521.674764] Lustre: Skipped 1 previous similar message [ 9525.467779] Lustre: server umount lustre-OST0000 complete [ 9531.872993] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9531.888901] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9531.891401] Lustre: Skipped 1 previous similar message [ 9535.647654] Lustre: server umount lustre-MDT0000 complete [ 9536.992539] LustreError: 304592:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9537.009162] LustreError: 304592:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 9539.435724] LustreError: 304572:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787382394 with bad export cookie 5607381880926131659 [ 9539.440160] LustreError: MGC192.168.202.147@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9539.452163] LustreError: 304572:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 9539.831465] Lustre: server umount lustre-MDT0001 complete [ 9552.231560] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing unload_modules_local [ 9555.497421] Key type lgssc unregistered [ 9555.824120] LNet: 308539:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 9555.839619] LNetError: 308539:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 9555.859257] LNet: Removed LNI 192.168.202.147@tcp [ 9556.864509] Key type .llcrypt unregistered [ 9556.869373] Key type ._llcrypt unregistered [ 9577.925834] Key type ._llcrypt registered [ 9577.928337] Key type .llcrypt registered [ 9578.040360] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_hostid [ 9592.457239] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing load_modules_local [ 9593.656644] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 9593.985755] alg: No test for adler32 (adler32-zlib) [ 9595.019879] Lustre: Lustre: Build Version: 2.17.57_47_gae0d133 [ 9595.283279] LNet: Added LNI 192.168.202.147@tcp [8/256/0/180] [ 9596.999165] Key type lgssc registered [ 9598.406419] Lustre: Echo OBD driver; http://www.lustre.org/ [ 9623.221698] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing load_modules_local [ 9639.880304] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 9639.902629] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9641.201900] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 9641.248622] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 9641.355916] Lustre: lustre-MDT0000: new disk, initializing [ 9641.448504] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9641.498492] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 9647.078703] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 9659.502954] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9659.860274] Lustre: lustre-OST0000: new disk, initializing [ 9659.869377] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 9659.882374] Lustre: Skipped 1 previous similar message [ 9659.891776] Lustre: 312586:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9660.019236] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9666.697668] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 9666.716049] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 9666.746723] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 9667.305936] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [10454.858774] Lustre: server umount lustre-MDT0000 complete [10458.774039] LustreError: 311771:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787383313 with bad export cookie 660106519590609366 [10458.781913] LustreError: MGC192.168.202.147@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10468.558953] Lustre: server umount lustre-OST0000 complete [10494.174434] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_hostid [10505.758733] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing load_modules_local [10541.938697] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10542.161342] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [10542.187987] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [10542.275241] Lustre: lustre-MDT0000: new disk, initializing [10542.344532] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10542.365495] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [10547.148673] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [10557.031295] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10557.216038] Lustre: 316913:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [10557.297252] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [10557.313910] Lustre: Skipped 1 previous similar message [10557.380173] Lustre: lustre-MDT0001: Not available for connect from 0@lo (not set up) [10557.423345] Lustre: lustre-MDT0001: new disk, initializing [10557.509516] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10557.538375] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [10557.543292] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [10561.303678] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [10571.692917] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [10577.823783] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [10584.775988] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10585.268236] Lustre: lustre-OST0000: new disk, initializing [10585.273984] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [10585.291757] Lustre: 318882:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [10585.386632] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10591.884400] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [10594.330422] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [10594.342615] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [10594.437485] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [10600.525750] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [10604.390508] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [10604.588277] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [10608.613503] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [10608.776839] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [10614.762867] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [10614.764128] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10614.766638] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10614.774364] LustreError: Skipped 1 previous similar message [10614.799833] Lustre: Skipped 1 previous similar message [10617.175827] Lustre: server umount lustre-OST0000 complete [10623.976838] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [10623.986943] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10623.997837] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10624.004870] Lustre: Skipped 1 previous similar message [10624.995106] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10627.538614] Lustre: server umount lustre-MDT0000 complete [10630.548294] LustreError: 317706:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787383485 with bad export cookie 660106519593550192 [10630.566874] LustreError: MGC192.168.202.147@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10630.578876] LustreError: 317706:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [10631.235625] Lustre: server umount lustre-MDT0001 complete [10634.593717] Lustre: DEBUG MARKER: SKIP: conf-sanity test_111 skipping SLOW test 111 [10636.042050] Lustre: DEBUG MARKER: == conf-sanity test 112a: mount OST with no_create option ========================================================== 03:24:50 (1787383490) [10641.881232] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10642.413454] LustreError: 321193:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10642.430080] LustreError: 321193:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [10642.532828] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10647.123534] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [10647.526827] LustreError: 321194:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10652.647291] LustreError: 321193:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10655.482740] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10655.762872] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10660.054996] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [10667.965870] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [10673.882887] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [10680.340165] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10680.585626] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10686.404126] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [10694.059303] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [10700.781309] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10700.978605] Lustre: lustre-OST0001: new disk, initializing [10700.985764] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [10700.993299] Lustre: 324029:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [10701.146974] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [10707.842715] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [10708.088185] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [10708.101893] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [10708.181619] LustreError: 324397:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [10709.219317] LustreError: 324397:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [10709.228311] LustreError: 324397:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 1 previous similar message [10710.243135] LustreError: 324397:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [10710.254834] LustreError: 324397:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 1 previous similar message [10712.305140] LustreError: 324397:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [10712.323348] LustreError: 324397:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 3 previous similar messages [10716.451078] LustreError: 324397:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [10716.461019] LustreError: 324397:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 8 previous similar messages [10717.917408] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [10718.144345] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [10723.102816] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff9b1382a7d800.ost_server_uuid 50 [10724.661885] LustreError: 324406:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [10724.683975] LustreError: 324406:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 14 previous similar messages [10724.753092] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff9b1382a7d800.ost_server_uuid in FULL state after 0 sec [10730.140770] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff9b1382a7d800.ost_server_uuid 50 [10731.357705] Lustre: DEBUG MARKER: osc.lustre-OST0001-osc-ffff9b1382a7d800.ost_server_uuid in FULL state after 0 sec [10738.185992] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [10748.385039] LustreError: lustre-OST0001-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [10748.386076] Lustre: lustre-OST0001-osc-MDT0001: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10748.392193] LustreError: Skipped 1 previous similar message [10748.393720] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [10748.409938] Lustre: Skipped 1 previous similar message [10748.425304] Lustre: Skipped 3 previous similar messages [10751.929917] Lustre: server umount lustre-OST0001 complete [10754.074528] LustreError: 323015:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 192.168.202.47@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [10759.653765] LustreError: 323014:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10759.663943] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [10759.670213] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10759.672689] LustreError: 323014:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [10759.673458] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10759.673462] Lustre: Skipped 3 previous similar messages [10764.141866] Lustre: server umount lustre-OST0000 complete [10768.360614] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [10768.370255] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10768.386253] Lustre: Skipped 1 previous similar message [10768.399249] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10768.410284] Lustre: Skipped 1 previous similar message [10774.223831] Lustre: server umount lustre-MDT0000 complete [10775.011357] LustreError: 321194:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10775.045044] LustreError: 321194:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [10778.095923] LustreError: 321174:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787383633 with bad export cookie 660106519593551067 [10778.102348] LustreError: MGC192.168.202.147@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10778.118634] LustreError: 321174:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [10778.590350] Lustre: server umount lustre-MDT0001 complete [10794.082611] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing unload_modules_local [10798.138849] Key type lgssc unregistered [10798.408849] LNet: 326719:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [10798.420836] LNetError: 326719:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [10798.458953] LNet: Removed LNI 192.168.202.147@tcp [10799.591333] Key type .llcrypt unregistered [10799.593617] Key type ._llcrypt unregistered [10827.309897] Key type ._llcrypt registered [10827.316241] Key type .llcrypt registered [10827.479486] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing load_modules_local [10828.886442] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [10828.907207] alg: No test for adler32 (adler32-zlib) [10829.961486] Lustre: Lustre: Build Version: 2.17.57_47_gae0d133 [10830.130639] LNet: Added LNI 192.168.202.147@tcp [8/256/0/180] [10831.792225] Key type lgssc registered [10833.339567] Lustre: Echo OBD driver; http://www.lustre.org/ [10840.502273] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [10840.524223] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10841.997175] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10847.805878] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [10851.300970] LustreError: 328142:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10851.348522] LustreError: 328142:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [10856.419103] LustreError: 328141:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10857.686566] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [10861.541129] LustreError: 328142:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10864.365248] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10864.747353] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10870.369372] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [10880.191538] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10880.574637] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10886.704710] LustreError: 329821:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10886.717766] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:65) [10887.472146] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [10894.821864] LustreError: 329820:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10894.837051] LustreError: 329820:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [10895.138395] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [10902.097424] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10902.410277] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [10907.653116] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:33) [10908.845871] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [10916.495957] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [10924.469954] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing wait_import_state (FULL|IDLE) os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [10924.753790] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [11025.376209] LustreError: lustre-OST0001-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [11025.384558] Lustre: lustre-OST0001-osc-MDT0000: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11025.385142] LustreError: Skipped 1 previous similar message [11025.385704] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [11025.438392] Lustre: Skipped 1 previous similar message [11030.496776] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [11030.503534] Lustre: Skipped 1 previous similar message [11035.624423] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [11035.628612] Lustre: Skipped 2 previous similar messages [11038.175203] Lustre: lustre-OST0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [11038.265915] Lustre: server umount lustre-OST0001 complete [11040.742085] LustreError: 331116:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11040.756373] LustreError: 331116:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [11045.344253] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [11045.351587] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11045.381877] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [11050.887834] Lustre: server umount lustre-OST0000 complete [11056.100694] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11056.104974] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11056.112334] Lustre: Skipped 2 previous similar messages [11056.121397] Lustre: Skipped 2 previous similar messages [11065.314279] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11065.324530] Lustre: Skipped 3 previous similar messages [11069.407192] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [11069.648677] Lustre: server umount lustre-MDT0000 complete [11070.432179] LustreError: 328138:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11070.445309] LustreError: 328138:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [11073.449704] LustreError: 331826:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787383928 with bad export cookie 13076336998447168408 [11073.456258] LustreError: MGC192.168.202.147@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11073.461594] LustreError: 331826:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [11073.775207] Lustre: server umount lustre-MDT0001 complete [11085.166837] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing unload_modules_local [11087.455715] Key type lgssc unregistered [11087.693646] LNet: 333092:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [11087.698827] LNetError: 333092:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [11087.712198] LNet: Removed LNI 192.168.202.147@tcp [11088.577176] Key type .llcrypt unregistered [11088.579205] Key type ._llcrypt unregistered [11136.460847] Key type ._llcrypt registered [11136.462727] Key type .llcrypt registered [11136.539343] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing load_modules_local [11137.645346] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [11137.658538] alg: No test for adler32 (adler32-zlib) [11138.689288] Lustre: Lustre: Build Version: 2.17.57_47_gae0d133 [11138.999989] LNet: Added LNI 192.168.202.147@tcp [8/256/0/180] [11140.663296] Key type lgssc registered [11142.047789] Lustre: Echo OBD driver; http://www.lustre.org/ [11151.919081] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [11151.951805] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11153.502670] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11158.285261] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [11163.625694] LustreError: 335385:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11163.659871] LustreError: 335385:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [11167.833651] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11168.135854] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11173.131405] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [11176.796524] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [11183.345399] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11183.757897] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [11190.284256] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [11194.027812] LustreError: 336877:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11198.148913] LustreError: 336877:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11198.176470] LustreError: 336877:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [11198.192836] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:97) [11200.597485] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11200.846260] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [11206.152799] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:65) [11208.522637] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [11218.240467] Lustre: DEBUG MARKER: Using TIMEOUT=20 [11223.883744] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [11236.842127] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11236.847339] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11236.865694] Lustre: Skipped 3 previous similar messages [11236.881164] Lustre: Skipped 3 previous similar messages [11242.006515] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11242.023620] Lustre: Skipped 3 previous similar messages [11243.139857] Lustre: server umount lustre-MDT0000 complete [11247.081836] LustreError: 336385:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11247.111936] LustreError: 336385:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [11247.475982] LustreError: 336123:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787384102 with bad export cookie 313314422712296261 [11247.488829] LustreError: 336123:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [11247.493467] LustreError: MGC192.168.202.147@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11248.227305] Lustre: server umount lustre-MDT0001 complete [11262.947485] Lustre: server umount lustre-OST0000 complete [11277.133942] Lustre: server umount lustre-OST0001 complete [11294.048219] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing load_modules_local [11304.678296] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11305.159898] LustreError: 340652:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11305.229917] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11310.119982] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [11315.685037] LustreError: 340653:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11315.725681] LustreError: 340653:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [11319.686771] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11320.222780] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11324.571582] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [11327.865844] Lustre: 341791:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [11334.344416] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11340.943467] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [11345.971032] LustreError: 342146:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11350.128090] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11350.471422] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [11350.482115] Lustre: Skipped 1 previous similar message [11351.495397] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:97) [11351.507312] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:129) [11356.199373] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [11363.574746] Lustre: DEBUG MARKER: Using TIMEOUT=20 [11367.141239] Lustre: 343662:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [11376.610135] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [11376.626467] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11376.657736] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11378.708428] Lustre: server umount lustre-MDT0000 complete [11382.245126] LustreError: 340649:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11382.273748] LustreError: 340649:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [11382.337613] LustreError: 340633:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787384237 with bad export cookie 313314422712297962 [11382.349605] LustreError: 340633:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [11382.363402] LustreError: MGC192.168.202.147@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11382.612475] Lustre: server umount lustre-MDT0001 complete [11396.718440] Lustre: server umount lustre-OST0000 complete [11411.346205] Lustre: server umount lustre-OST0001 complete [11415.297831] Lustre: DEBUG MARKER: SKIP: conf-sanity test_114 skipping SLOW test 114 [11417.210686] Lustre: DEBUG MARKER: SKIP: conf-sanity test_115 skipping excluded test 115 [11419.246558] Lustre: DEBUG MARKER: == conf-sanity test 116: big size MDT support ============ 03:37:53 (1787384273) [11444.903605] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing load_modules_local [11454.495850] loop0: detected capacity change from 0 to 1099511627776 [11455.197580] SGI XFS with ACLs, security attributes, quota, no debug enabled [11455.212041] *** VALIDATE xfs *** [11455.238913] XFS (loop0): Mounting V5 Filesystem [11455.276961] XFS (loop0): Ending clean mount [11456.669188] loop1: detected capacity change from 0 to 18691697672192 [11574.862316] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_hostid [11586.168586] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing load_modules_local [11626.182271] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11626.575239] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [11626.607119] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [11626.729723] Lustre: lustre-MDT0000: new disk, initializing [11626.859486] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11626.915620] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [11633.851840] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [11649.542250] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11649.774466] Lustre: 348798:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [11649.925340] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [11649.943993] Lustre: Skipped 1 previous similar message [11650.095842] Lustre: lustre-MDT0001: new disk, initializing [11650.350788] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [11650.365296] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [11655.482646] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [11666.180117] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [11672.622893] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [11679.936435] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11680.313044] Lustre: lustre-OST0000: new disk, initializing [11680.317515] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [11680.332784] Lustre: 350768:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [11682.081839] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [11682.094788] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [11682.269409] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [11687.269870] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [11697.215914] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [11702.714732] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [11703.033857] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [11708.463529] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [11708.708433] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [11712.994056] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11712.997780] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [11713.015613] Lustre: Skipped 4 previous similar messages [11713.019795] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [11713.020343] Lustre: Skipped 4 previous similar messages [11718.127848] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [11718.132362] Lustre: Skipped 1 previous similar message [11725.279929] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [11725.590880] Lustre: server umount lustre-OST0000 complete [11732.449345] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [11732.458937] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11732.476461] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11732.488025] Lustre: Skipped 3 previous similar messages [11744.224754] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [11744.395435] Lustre: server umount lustre-MDT0000 complete [11748.540426] LustreError: 348791:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787384603 with bad export cookie 313314422712299509 [11748.560379] LustreError: MGC192.168.202.147@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11748.563989] LustreError: 348791:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 8 previous similar messages [11749.108182] Lustre: server umount lustre-MDT0001 complete [11760.490298] XFS (loop0): Unmounting Filesystem [11766.724883] Lustre: DEBUG MARKER: == conf-sanity test 117: lctl get_param return errors properly ========================================================== 03:43:39 (1787384619) [11776.371734] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11777.261070] LustreError: 353332:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11777.390553] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11777.399619] Lustre: Skipped 2 previous similar messages [11782.569703] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [11791.846669] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11797.643969] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [11807.831896] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [11814.085553] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [11821.474162] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11828.648890] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [11837.847651] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [11847.652062] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [11847.652837] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11847.663105] Lustre: Skipped 1 previous similar message [11847.674338] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [11847.679102] Lustre: Skipped 6 previous similar messages [11854.090959] Lustre: server umount lustre-OST0000 complete [11858.915962] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [11858.925576] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11858.945866] Lustre: Skipped 1 previous similar message [11864.527056] Lustre: server umount lustre-MDT0000 complete [11869.043239] LustreError: 353314:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787384723 with bad export cookie 313314422712300398 [11869.050330] LustreError: MGC192.168.202.147@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11869.054882] LustreError: 353314:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [11869.698610] Lustre: server umount lustre-MDT0001 complete [11885.036251] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing unload_modules_local [11888.720407] Key type lgssc unregistered [11889.263406] LNet: 356967:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [11889.277701] LNetError: 356967:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [11889.315526] LNet: Removed LNI 192.168.202.147@tcp [11890.432243] Key type .llcrypt unregistered [11890.433945] Key type ._llcrypt unregistered [11924.007889] Key type ._llcrypt registered [11924.010358] Key type .llcrypt registered [11924.166502] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing load_modules_local [11926.378523] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [11926.590871] alg: No test for adler32 (adler32-zlib) [11927.918868] Lustre: Lustre: Build Version: 2.17.57_47_gae0d133 [11928.351135] LNet: Added LNI 192.168.202.147@tcp [8/256/0/180] [11930.183757] Key type lgssc registered [11931.966970] Lustre: Echo OBD driver; http://www.lustre.org/ [11940.481283] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [11940.510818] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11942.117035] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11947.493564] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [11952.612676] LustreError: 358534:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11952.660224] LustreError: 358534:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [11957.730347] LustreError: 358535:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11958.022410] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11958.532733] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11963.121829] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [11971.269548] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [11976.512811] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [11983.679942] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11984.129245] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [11991.195918] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [12000.000567] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [12013.127067] Lustre: lustre-MDT0001: Not available for connect from 192.168.202.47@tcp (stopping) [12014.049106] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12014.071574] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [12014.073720] Lustre: Skipped 1 previous similar message [12014.104419] Lustre: Skipped 1 previous similar message [12017.247933] Lustre: server umount lustre-MDT0001 complete [12018.212694] LustreError: 358531:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.202.47@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [12023.313445] LustreError: 358531:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.202.47@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [12023.335499] LustreError: 358531:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [12028.449204] LustreError: 358529:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.202.47@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [12028.472701] LustreError: 358529:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [12031.677729] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12031.854192] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [12031.862734] Lustre: Found index 1 for lustre-MDT0001, updating log [12031.947726] Lustre: 358524:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [12032.290214] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [12037.621833] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [12037.882377] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [12038.158352] LustreError: 361992:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [12038.170094] LustreError: 361992:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [12038.189494] LustreError: 358526:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [12038.332277] LustreError: 362003:0:(obd_config.c:2102:class_config_llog_handler()) MGC192.168.202.147@tcp: cfg command failed: rc = -17 [12038.359227] Lustre: cmd=cf001 0:lustre-MDT0001-osp-MDT0000 1:osp 2:lustre-MDT0000-mdtlov_UUID [12038.359227] [12052.960460] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [12052.964210] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12052.971321] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [12052.978696] Lustre: Skipped 1 previous similar message [12053.017322] Lustre: Skipped 1 previous similar message [12058.081540] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [12058.098124] Lustre: Skipped 2 previous similar messages [12063.205582] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [12063.227382] Lustre: Skipped 2 previous similar messages [12065.247872] Lustre: lustre-MDT0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [12065.440760] Lustre: server umount lustre-MDT0001 complete [12068.320799] LustreError: 358535:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12068.336353] LustreError: 358535:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [12076.323690] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12076.415060] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [12076.418375] Lustre: Found index 1 for lustre-MDT0001, updating log [12076.446101] Lustre: 358521:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [12076.726188] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [12082.181249] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [12082.213426] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [12086.279491] LustreError: 363213:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [12086.293156] LustreError: 363213:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [12086.304298] LustreError: 358526:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [12086.311094] LustreError: 358526:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [12086.396971] LustreError: 363216:0:(obd_config.c:761:class_setup()) osp: device 8 already setup: rc = -17 [12086.400548] LustreError: 363216:0:(obd_config.c:866:class_detach()) OBD device 8 still set up [12086.429704] LustreError: 363216:0:(obd_config.c:2102:class_config_llog_handler()) MGC192.168.202.147@tcp: cfg command failed: rc = -17 [12086.444869] Lustre: cmd=cf003 0:lustre-MDT0001-osp-MDT0000 1:lustre-MDT0001_UUID 2:192.168.202.147@tcp [12086.444869] [12389.349939] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [12389.354455] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12389.355601] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [12389.408619] Lustre: Skipped 2 previous similar messages [12394.898306] Lustre: server umount lustre-MDT0001 complete [12397.073836] LustreError: 362239:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.202.47@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [12397.103417] LustreError: 362239:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 4 previous similar messages [12408.141532] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12408.298943] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [12408.303172] Lustre: Found index 1 for lustre-MDT0001, updating log [12408.348942] Lustre: 358522:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [12408.801031] Lustre: lustre-MDT0001: Imperative Recovery enabled, recovery window shrunk from 60-180 down to 60-180 [12413.960546] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [12414.134986] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [12415.032060] LustreError: 371566:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [12415.044908] LustreError: 371566:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [12415.065207] LustreError: 358526:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [12415.073562] LustreError: 358526:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [12415.189713] LustreError: 371575:0:(lod_lov.c:208:lod_add_device()) lustre-MDT0001-osp-MDT0000: cannot connect to next dev lustre-MDT0001_UUID (-114) [12415.204669] LustreError: 371575:0:(obd_config.c:2102:class_config_llog_handler()) MGC192.168.202.147@tcp: cfg command failed: rc = -114 [12415.212236] Lustre: cmd=cf014 0:lustre-MDT0000-mdtlov 1:lustre-MDT0001_UUID 2:1 3:1 [12415.212236] [12739.554412] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12739.565101] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12739.581647] Lustre: Skipped 3 previous similar messages [12739.603715] Lustre: Skipped 5 previous similar messages [12743.068567] Lustre: server umount lustre-MDT0000 complete [12744.697367] LustreError: 362239:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12744.721342] LustreError: 362239:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 8 previous similar messages [12746.692763] LustreError: 361722:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787385601 with bad export cookie 4019891665273660084 [12746.696823] LustreError: MGC192.168.202.147@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [12746.699732] LustreError: 361722:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [12749.796529] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12752.998759] Lustre: server umount lustre-MDT0001 complete [12757.056776] Lustre: server umount lustre-OST0000 complete [12766.082120] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_hostid [12777.766400] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing load_modules_local [12815.211195] Lustre: DEBUG MARKER: == conf-sanity test 120: cross-target rename should not create bad symlinks ========================================================== 04:01:09 (1787385669) [12821.552179] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12821.758310] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [12821.795456] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [12821.899806] Lustre: lustre-MDT0000: new disk, initializing [12821.980279] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [12821.996319] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [12826.621817] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [12837.480369] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12837.599152] Lustre: 382382:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [12837.639705] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [12837.642523] Lustre: Skipped 1 previous similar message [12837.697805] Lustre: lustre-MDT0001: Not available for connect from 0@lo (not set up) [12837.701699] Lustre: Skipped 2 previous similar messages [12837.730622] Lustre: lustre-MDT0001: new disk, initializing [12837.825711] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [12837.868496] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [12837.884320] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [12842.745755] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [12853.406144] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [12859.350325] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [12867.272274] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12867.628165] Lustre: lustre-OST0000: new disk, initializing [12867.637985] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [12867.649743] Lustre: 384348:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [12867.781854] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [12873.325840] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [12873.348411] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [12873.512289] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [12875.584702] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [12887.595586] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [12897.610115] LustreError: lustre-MDT0000-osp-MDT0001: operation out_update to node 0@lo failed: rc = -107 [12897.625242] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12898.154660] Lustre: server umount lustre-MDT0000 complete [12899.300185] LustreError: 382388:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12899.308471] LustreError: 382388:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [12902.034750] LustreError: 382375:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787385756 with bad export cookie 4019891665273662037 [12902.052865] LustreError: 383192:0:(osp_object.c:618:osp_attr_get()) lustre-MDT0000-osp-MDT0001: osp_attr_get update error [0x200000401:0x2:0x0]: rc = -5 [12902.056084] LustreError: MGC192.168.202.147@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [12902.069662] LustreError: 383192:0:(client.c:1394:ptlrpc_import_delay_req()) @@@ IMP_CLOSED req@00000000eecf3253 x1874208786536192/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 264/4320 e 0 to 0 dl 0 ref 2 fl Rpc:QU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [12902.717975] Lustre: server umount lustre-MDT0001 complete [12919.214913] Lustre: server umount lustre-OST0000 complete [12961.942298] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_hostid [12979.970167] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing load_modules_local [13019.331478] Lustre: DEBUG MARKER: == conf-sanity test complete, duration 12623 sec ========= 04:04:32 (1787385872) [13021.270770] Lustre: DEBUG MARKER: === conf-sanity: start cleanup 04:04:35 (1787385875) === [13024.861145] Lustre: DEBUG MARKER: === conf-sanity: finish cleanup 04:04:38 (1787385878) === [13069.614734] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing unload_modules_local [13072.611993] Key type lgssc unregistered [13073.029090] LNet: 390744:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [13073.037868] LNetError: 390744:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [13073.058926] LNet: Removed LNI 192.168.202.147@tcp [13074.049157] Key type .llcrypt unregistered [13074.050583] Key type ._llcrypt unregistered