[ 0.000000] Linux version 4.18.0rh8.10-debug (green@maintenance) (gcc version 8.5.0 20210514 (Red Hat 8.5.0-26) (GCC)) #2 SMP Mon Jul 14 01:24:22 EDT 2025 [ 0.000000] Command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers' [ 0.000000] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256 [ 0.000000] x86/fpu: Enabled xstate features 0x7, context size is 832 bytes, using 'standard' format. [ 0.000000] signal: max sigframe size: 1776 [ 0.000000] BIOS-provided physical RAM map: [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x00000000bffcdfff] usable [ 0.000000] BIOS-e820: [mem 0x00000000bffce000-0x00000000bfffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x0000000146dfffff] usable [ 0.000000] NX (Execute Disable) protection: active [ 0.000000] SMBIOS 3.0.0 present. [ 0.000000] DMI: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.16.3-2.fc40 04/01/2014 [ 0.000000] Hypervisor detected: KVM [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00 [ 0.000000] kvm-clock: using sched offset of 443113217 cycles [ 0.000000] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns [ 0.000000] tsc: Detected 2399.998 MHz processor [ 0.000000] last_pfn = 0x146e00 max_arch_pfn = 0x400000000 [ 0.000000] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT [ 0.000000] last_pfn = 0xbffce max_arch_pfn = 0x400000000 [ 0.000000] found SMP MP-table at [mem 0x000f53f0-0x000f53ff] [ 0.000000] RAMDISK: [mem 0xbcc54000-0xbffbffff] [ 0.000000] ACPI: Early table checksum verification disabled [ 0.000000] ACPI: RSDP 0x00000000000F5200 000014 (v00 BOCHS ) [ 0.000000] ACPI: RSDT 0x00000000BFFE1D87 000034 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACP 0x00000000BFFE1C23 000074 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: DSDT 0x00000000BFFE0040 001BE3 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACS 0x00000000BFFE0000 000040 [ 0.000000] ACPI: APIC 0x00000000BFFE1C97 000090 (v03 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: HPET 0x00000000BFFE1D27 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: WAET 0x00000000BFFE1D5F 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: Reserving FACP table memory at [mem 0xbffe1c23-0xbffe1c96] [ 0.000000] ACPI: Reserving DSDT table memory at [mem 0xbffe0040-0xbffe1c22] [ 0.000000] ACPI: Reserving FACS table memory at [mem 0xbffe0000-0xbffe003f] [ 0.000000] ACPI: Reserving APIC table memory at [mem 0xbffe1c97-0xbffe1d26] [ 0.000000] ACPI: Reserving HPET table memory at [mem 0xbffe1d27-0xbffe1d5e] [ 0.000000] ACPI: Reserving WAET table memory at [mem 0xbffe1d5f-0xbffe1d86] [ 0.000000] No NUMA configuration found [ 0.000000] Faking a node at [mem 0x0000000000000000-0x0000000146dfffff] [ 0.000000] NODE_DATA(0) allocated [mem 0x1465a3000-0x1465cdfff] [ 0.000000] Reserving 256MB of memory at 2752MB for crashkernel (System RAM: 4205MB) [ 0.000000] Zone ranges: [ 0.000000] DMA [mem 0x0000000000001000-0x0000000000ffffff] [ 0.000000] DMA32 [mem 0x0000000001000000-0x00000000ffffffff] [ 0.000000] Normal [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Device empty [ 0.000000] Movable zone start for each node [ 0.000000] Early memory node ranges [ 0.000000] node 0: [mem 0x0000000000001000-0x000000000009efff] [ 0.000000] node 0: [mem 0x0000000000100000-0x00000000bffcdfff] [ 0.000000] node 0: [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Zeroed struct page in unavailable ranges: 4756 pages [ 0.000000] Initmem setup node 0 [mem 0x0000000000001000-0x0000000146dfffff] [ 0.000000] ACPI: PM-Timer IO Port: 0x608 [ 0.000000] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1]) [ 0.000000] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23 [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level) [ 0.000000] Using ACPI (MADT) for SMP configuration information [ 0.000000] ACPI: HPET id: 0x8086a201 base: 0xfed00000 [ 0.000000] TSC deadline timer available [ 0.000000] smpboot: Allowing 4 CPUs, 0 hotplug CPUs [ 0.000000] kvm-guest: KVM setup pv remote TLB flush [ 0.000000] kvm-guest: setup PV sched yield [ 0.000000] PM: Registered nosave memory: [mem 0x00000000-0x00000fff] [ 0.000000] PM: Registered nosave memory: [mem 0x0009f000-0x0009ffff] [ 0.000000] PM: Registered nosave memory: [mem 0x000a0000-0x000effff] [ 0.000000] PM: Registered nosave memory: [mem 0x000f0000-0x000fffff] [ 0.000000] PM: Registered nosave memory: [mem 0xbffce000-0xbfffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xc0000000-0xfeffbfff] [ 0.000000] PM: Registered nosave memory: [mem 0xfeffc000-0xfeffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xff000000-0xfffbffff] [ 0.000000] PM: Registered nosave memory: [mem 0xfffc0000-0xffffffff] [ 0.000000] [mem 0xc0000000-0xfeffbfff] available for PCI devices [ 0.000000] Booting paravirtualized kernel on KVM [ 0.000000] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns [ 0.000000] setup_percpu: NR_CPUS:8192 nr_cpumask_bits:4 nr_cpu_ids:4 nr_node_ids:1 [ 0.000000] percpu: Embedded 63 pages/cpu s221184 r8192 d28672 u524288 [ 0.000000] kvm-guest: PV spinlocks enabled [ 0.000000] PV qspinlock hash table entries: 256 (order: 0, 4096 bytes, linear) [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 1059606 [ 0.000000] Policy zone: Normal [ 0.000000] Kernel command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] Specific versions of hardware are certified with Red Hat Enterprise Linux 8. Please see the list of hardware certified with Red Hat Enterprise Linux 8 at https://catalog.redhat.com. [ 0.000000] audit: disabled (until reboot) [ 0.000000] software IO TLB: area num 4. [ 0.000000] Memory: 2829652K/4306352K available (18435K kernel code, 11221K rwdata, 7248K rodata, 2908K init, 18040K bss, 524580K reserved, 0K cma-reserved) [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1 [ 0.000000] kmemleak: Kernel memory leak detector disabled [ 0.000000] ftrace: allocating 41240 entries in 162 pages [ 0.000000] ftrace: allocated 162 pages with 3 groups [ 0.000000] rcu: Hierarchical RCU implementation. [ 0.000000] rcu: RCU event tracing is enabled. [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=8192 to nr_cpu_ids=4. [ 0.000000] rcu: RCU callback double-/use-after-free debug enabled. [ 0.000000] Rude variant of Tasks RCU enabled. [ 0.000000] Tracing variant of Tasks RCU enabled. [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies. [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4 [ 0.000000] NR_IRQS: 524544, nr_irqs: 456, preallocated irqs: 16 [ 0.000000] random: get_random_bytes called from start_kernel+0x622/0x9a8 with crng_init=0 [ 0.001000] Console: colour *CGA 80x25 [ 0.001000] printk: console [ttyS1] enabled [ 0.001000] ACPI: Core revision 20220331 [ 0.001000] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns [ 0.001013] APIC: Switch to symmetric I/O mode setup [ 0.003167] x2apic enabled [ 0.004006] Switched APIC routing to physical x2apic. [ 0.005008] kvm-guest: setup PV IPIs [ 0.007800] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1 [ 0.008000] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 0.008010] Calibrating delay loop (skipped) preset value.. 4799.99 BogoMIPS (lpj=2399998) [ 0.009005] pid_max: default: 32768 minimum: 301 [ 0.010095] LSM: Security Framework initializing [ 0.011024] Yama: becoming mindful. [ 0.011631] SELinux: Initializing. [ 0.012049] *** VALIDATE selinux *** [ 0.019054] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, vmalloc) [ 0.024112] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, vmalloc) [ 0.025105] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.026131] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.027113] *** VALIDATE tmpfs *** [ 0.028454] *** VALIDATE proc *** [ 0.029260] *** VALIDATE cgroup *** [ 0.030005] *** VALIDATE cgroup2 *** [ 0.031222] x86/cpu: User Mode Instruction Prevention (UMIP) activated [ 0.032133] Last level iTLB entries: 4KB 0, 2MB 0, 4MB 0 [ 0.033003] Last level dTLB entries: 4KB 0, 2MB 0, 4MB 0, 1GB 0 [ 0.034025] Spectre V2 : User space: Vulnerable [ 0.035004] Speculative Store Bypass: Vulnerable [ 0.037608] debug: unmapping init [mem 0xffffffffa0259000-0xffffffffa0260fff] [ 0.039121] smpboot: CPU0: Intel(R) Xeon(R) CPU E5-2695 v2 @ 2.40GHz (family: 0x6, model: 0x3e, stepping: 0x4) [ 0.040420] Performance Events: IvyBridge events, full-width counters, Intel PMU driver. [ 0.041010] ... version: 2 [ 0.041772] ... bit width: 48 [ 0.042005] ... generic registers: 4 [ 0.042677] ... value mask: 0000ffffffffffff [ 0.043006] ... max period: 00007fffffffffff [ 0.043949] ... fixed-purpose events: 3 [ 0.044005] ... event mask: 000000070000000f [ 0.045024] rcu: Hierarchical SRCU implementation. [ 0.046896] smp: Bringing up secondary CPUs ... [ 0.047482] x86: Booting SMP configuration: [ 0.048012] .... node #0, CPUs: #1 #2 #3 [ 0.050663] smp: Brought up 1 node, 4 CPUs [ 0.051689] smpboot: Max logical packages: 1 [ 0.052005] smpboot: Total of 4 processors activated (19199.98 BogoMIPS) [ 0.200028] node 0 deferred pages initialised in 147ms [ 0.203441] devtmpfs: initialized [ 0.204262] x86/mm: Memory block size: 128MB [ 0.206612] gcov: version magic: 0x41383552 [ 0.209345] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns [ 0.210073] futex hash table entries: 1024 (order: 4, 65536 bytes, vmalloc) [ 0.212390] pinctrl core: initialized pinctrl subsystem [ 0.213115] [ 0.213570] ************************************************************* [ 0.215009] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.216008] ** ** [ 0.218009] ** IOMMU DebugFS SUPPORT HAS BEEN ENABLED IN THIS KERNEL ** [ 0.219005] ** ** [ 0.221005] ** This means that this kernel is built to expose internal ** [ 0.222011] ** IOMMU data structures, which may compromise security on ** [ 0.223006] ** your system. ** [ 0.225009] ** ** [ 0.227006] ** If you see this message and you are not debugging the ** [ 0.228005] ** kernel, report this immediately to your vendor! ** [ 0.230009] ** ** [ 0.233014] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.235007] ************************************************************* [ 0.237496] NET: Registered protocol family 16 [ 0.238303] DMA: preallocated 512 KiB GFP_KERNEL pool for atomic allocations [ 0.240054] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations [ 0.241074] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations [ 0.245100] cpuidle: using governor menu [ 0.246287] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5 [ 0.248391] PCI: Using configuration type 1 for base access [ 0.250104] core: PMU erratum BJ122, BV98, HSD29 worked around, HT is on [ 0.256111] HugeTLB registered 1.00 GiB page size, pre-allocated 0 pages [ 0.257011] HugeTLB registered 2.00 MiB page size, pre-allocated 0 pages [ 0.258105] cryptd: max_cpu_qlen set to 1000 [ 0.259179] ACPI: Added _OSI(Module Device) [ 0.261007] ACPI: Added _OSI(Processor Device) [ 0.261955] ACPI: Added _OSI(3.0 _SCP Extensions) [ 0.263008] ACPI: Added _OSI(Processor Aggregator Device) [ 0.267960] ACPI: 1 ACPI AML tables successfully acquired and loaded [ 0.274350] ACPI: Interpreter enabled [ 0.275036] ACPI: PM: (supports S0 S3 S4 S5) [ 0.276010] ACPI: Using IOAPIC for interrupt routing [ 0.277052] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug [ 0.280359] ACPI: Enabled 2 GPEs in block 00 to 0F [ 0.287184] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff]) [ 0.289017] acpi PNP0A03:00: _OSC: OS supports [ASPM ClockPM Segments MSI HPX-Type3] [ 0.291014] acpi PNP0A03:00: _OSC: not requesting OS control; OS requires [ExtendedConfig ASPM ClockPM MSI] [ 0.293067] acpi PNP0A03:00: fail to add MMCONFIG information, can't access extended PCI configuration space under this bridge. [ 0.297417] acpiphp: Slot [2] registered [ 0.298094] acpiphp: Slot [5] registered [ 0.299081] acpiphp: Slot [6] registered [ 0.301068] acpiphp: Slot [7] registered [ 0.302096] acpiphp: Slot [8] registered [ 0.303085] acpiphp: Slot [9] registered [ 0.305102] acpiphp: Slot [10] registered [ 0.306082] acpiphp: Slot [3] registered [ 0.307064] acpiphp: Slot [4] registered [ 0.308063] acpiphp: Slot [11] registered [ 0.310063] acpiphp: Slot [12] registered [ 0.311075] acpiphp: Slot [13] registered [ 0.312089] acpiphp: Slot [14] registered [ 0.314063] acpiphp: Slot [15] registered [ 0.315076] acpiphp: Slot [16] registered [ 0.316061] acpiphp: Slot [17] registered [ 0.318059] acpiphp: Slot [18] registered [ 0.319063] acpiphp: Slot [19] registered [ 0.320077] acpiphp: Slot [20] registered [ 0.322061] acpiphp: Slot [21] registered [ 0.323063] acpiphp: Slot [22] registered [ 0.324081] acpiphp: Slot [23] registered [ 0.325061] acpiphp: Slot [24] registered [ 0.327063] acpiphp: Slot [25] registered [ 0.328059] acpiphp: Slot [26] registered [ 0.329081] acpiphp: Slot [27] registered [ 0.330064] acpiphp: Slot [28] registered [ 0.332062] acpiphp: Slot [29] registered [ 0.333058] acpiphp: Slot [30] registered [ 0.334059] acpiphp: Slot [31] registered [ 0.335039] PCI host bridge to bus 0000:00 [ 0.336012] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window] [ 0.338012] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window] [ 0.340028] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window] [ 0.343013] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window] [ 0.345025] pci_bus 0000:00: root bus resource [mem 0x380000000000-0x38007fffffff window] [ 0.348039] pci_bus 0000:00: root bus resource [bus 00-ff] [ 0.350202] pci 0000:00:00.0: [8086:1237] type 00 class 0x060000 [ 0.351633] pci 0000:00:01.0: [8086:7000] type 00 class 0x060100 [ 0.354055] pci 0000:00:01.1: [8086:7010] type 00 class 0x010180 [ 0.363021] pci 0000:00:01.1: reg 0x20: [io 0xc320-0xc32f] [ 0.368064] pci 0000:00:01.1: legacy IDE quirk: reg 0x10: [io 0x01f0-0x01f7] [ 0.370025] pci 0000:00:01.1: legacy IDE quirk: reg 0x14: [io 0x03f6] [ 0.373024] pci 0000:00:01.1: legacy IDE quirk: reg 0x18: [io 0x0170-0x0177] [ 0.374022] pci 0000:00:01.1: legacy IDE quirk: reg 0x1c: [io 0x0376] [ 0.376594] pci 0000:00:01.3: [8086:7113] type 00 class 0x068000 [ 0.378598] pci 0000:00:01.3: quirk: [io 0x0600-0x063f] claimed by PIIX4 ACPI [ 0.381044] pci 0000:00:01.3: quirk: [io 0x0700-0x070f] claimed by PIIX4 SMB [ 0.383794] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 [ 0.387013] pci 0000:00:02.0: reg 0x10: [io 0xc300-0xc31f] [ 0.395875] pci 0000:00:02.0: reg 0x20: [mem 0x380000000000-0x380000003fff 64bit pref] [ 0.400012] pci 0000:00:02.0: reg 0x30: [mem 0xfeb80000-0xfebbffff pref] [ 0.404430] pci 0000:00:05.0: [1af4:1001] type 00 class 0x010000 [ 0.412028] pci 0000:00:05.0: reg 0x10: [io 0xc000-0xc07f] [ 0.419023] pci 0000:00:05.0: reg 0x14: [mem 0xfebc0000-0xfebc0fff] [ 0.434014] pci 0000:00:05.0: reg 0x20: [mem 0x380000004000-0x380000007fff 64bit pref] [ 0.442150] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 [ 0.447013] pci 0000:00:06.0: reg 0x10: [io 0xc080-0xc0ff] [ 0.453009] pci 0000:00:06.0: reg 0x14: [mem 0xfebc1000-0xfebc1fff] [ 0.466013] pci 0000:00:06.0: reg 0x20: [mem 0x380000008000-0x38000000bfff 64bit pref] [ 0.474952] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 [ 0.479022] pci 0000:00:07.0: reg 0x10: [io 0xc100-0xc17f] [ 0.484019] pci 0000:00:07.0: reg 0x14: [mem 0xfebc2000-0xfebc2fff] [ 0.496018] pci 0000:00:07.0: reg 0x20: [mem 0x38000000c000-0x38000000ffff 64bit pref] [ 0.507198] pci 0000:00:08.0: [1af4:1001] type 00 class 0x010000 [ 0.514025] pci 0000:00:08.0: reg 0x10: [io 0xc180-0xc1ff] [ 0.518015] pci 0000:00:08.0: reg 0x14: [mem 0xfebc3000-0xfebc3fff] [ 0.531014] pci 0000:00:08.0: reg 0x20: [mem 0x380000010000-0x380000013fff 64bit pref] [ 0.537397] pci 0000:00:09.0: [1af4:1001] type 00 class 0x010000 [ 0.544013] pci 0000:00:09.0: reg 0x10: [io 0xc200-0xc27f] [ 0.549011] pci 0000:00:09.0: reg 0x14: [mem 0xfebc4000-0xfebc4fff] [ 0.558013] pci 0000:00:09.0: reg 0x20: [mem 0x380000014000-0x380000017fff 64bit pref] [ 0.568557] pci 0000:00:0a.0: [1af4:1001] type 00 class 0x010000 [ 0.575020] pci 0000:00:0a.0: reg 0x10: [io 0xc280-0xc2ff] [ 0.580019] pci 0000:00:0a.0: reg 0x14: [mem 0xfebc5000-0xfebc5fff] [ 0.591015] pci 0000:00:0a.0: reg 0x20: [mem 0x380000018000-0x38000001bfff 64bit pref] [ 0.600213] ACPI: PCI: Interrupt link LNKA configured for IRQ 10 [ 0.603329] ACPI: PCI: Interrupt link LNKB configured for IRQ 10 [ 0.604289] ACPI: PCI: Interrupt link LNKC configured for IRQ 11 [ 0.606220] ACPI: PCI: Interrupt link LNKD configured for IRQ 11 [ 0.607103] ACPI: PCI: Interrupt link LNKS configured for IRQ 9 [ 0.610379] iommu: Default domain type: Passthrough [ 0.612345] SCSI subsystem initialized [ 0.614121] ACPI: bus type USB registered [ 0.615159] usbcore: registered new interface driver usbfs [ 0.617069] usbcore: registered new interface driver hub [ 0.619084] usbcore: registered new device driver usb [ 0.620132] pps_core: LinuxPPS API ver. 1 registered [ 0.622008] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti [ 0.624043] PTP clock support registered [ 0.626097] EDAC MC: Ver: 3.0.0 [ 0.627436] PCI: Using ACPI for IRQ routing [ 0.629691] NetLabel: Initializing [ 0.631009] NetLabel: domain hash size = 128 [ 0.632006] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO [ 0.633050] NetLabel: unlabeled traffic allowed by default [ 0.635090] vgaarb: loaded [ 0.636355] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0 [ 0.638011] hpet0: 3 comparators, 64-bit 100.000000 MHz counter [ 0.643003] clocksource: Switched to clocksource kvm-clock [ 0.747315] VFS: Disk quotas dquot_6.6.0 [ 0.748893] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) [ 0.750704] *** VALIDATE ramfs *** [ 0.751786] *** VALIDATE hugetlbfs *** [ 0.752971] pnp: PnP ACPI init [ 0.754532] pnp: PnP ACPI: found 6 devices [ 0.769632] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns [ 0.772244] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window] [ 0.774037] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window] [ 0.776097] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window] [ 0.778442] pci_bus 0000:00: resource 7 [mem 0xc0000000-0xfebfffff window] [ 0.780124] pci_bus 0000:00: resource 8 [mem 0x380000000000-0x38007fffffff window] [ 0.781900] NET: Registered protocol family 2 [ 0.784145] IP idents hash table entries: 131072 (order: 8, 1048576 bytes, vmalloc) [ 0.787405] tcp_listen_portaddr_hash hash table entries: 4096 (order: 5, 163840 bytes, vmalloc) [ 0.790495] TCP established hash table entries: 65536 (order: 7, 524288 bytes, vmalloc) [ 0.795793] TCP bind hash table entries: 65536 (order: 9, 2097152 bytes, vmalloc) [ 0.798698] TCP: Hash tables configured (established 65536 bind 65536) [ 0.801367] MPTCP token hash table entries: 8192 (order: 6, 393216 bytes, vmalloc) [ 0.803867] UDP hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 0.806179] UDP-Lite hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 0.809155] NET: Registered protocol family 1 [ 0.811272] RPC: Registered named UNIX socket transport module. [ 0.812981] RPC: Registered udp transport module. [ 0.814451] RPC: Registered tcp transport module. [ 0.815905] RPC: Registered tcp NFSv4.1 backchannel transport module. [ 0.817973] NET: Registered protocol family 44 [ 0.819321] pci 0000:00:00.0: Limiting direct PCI/PCI transfers [ 0.821169] pci 0000:00:01.0: PIIX3: Enabling Passive Release [ 0.822930] pci 0000:00:01.0: Activating ISA DMA hang workarounds [ 0.825031] PCI: CLS 0 bytes, default 64 [ 0.826545] Unpacking initramfs... [ 2.284300] debug: unmapping init [mem 0xffff89763cc54000-0xffff89763ffbffff] [ 2.290148] PCI-DMA: Using software bounce buffering for IO (SWIOTLB) [ 2.293181] software IO TLB: mapped [mem 0x00000000a8000000-0x00000000ac000000] (64MB) [ 2.295393] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 2.845506] Initialise system trusted keyrings [ 2.847356] Key type blacklist registered [ 2.849711] workingset: timestamp_bits=36 max_order=20 bucket_order=0 [ 2.859488] zbud: loaded [ 2.863631] *** VALIDATE nfs *** [ 2.865033] *** VALIDATE nfs4 *** [ 2.866661] pstore: using deflate compression [ 2.870319] Platform Keyring initialized [ 3.017993] NET: Registered protocol family 38 [ 3.019393] Key type asymmetric registered [ 3.020552] Asymmetric key parser 'x509' registered [ 3.021950] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 247) [ 3.024331] io scheduler mq-deadline registered [ 3.025729] io scheduler kyber registered [ 3.027106] io scheduler bfq registered [ 3.031657] atomic64_test: passed for x86-64 platform with CX8 and with SSE [ 3.034587] shpchp: Standard Hot Plug PCI Controller Driver version: 0.4 [ 3.037305] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0 [ 3.040507] ACPI: Power Button [PWRF] [ 3.142419] ACPI: \_SB_.LNKB: Enabled at IRQ 10 [ 3.244560] ACPI: \_SB_.LNKA: Enabled at IRQ 11 [ 3.429728] ACPI: \_SB_.LNKC: Enabled at IRQ 11 [ 3.531498] ACPI: \_SB_.LNKD: Enabled at IRQ 10 [ 3.738178] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled [ 3.765825] 00:03: ttyS1 at I/O 0x2f8 (irq = 3, base_baud = 115200) is a 16550A [ 3.799432] 00:04: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A [ 3.806664] Non-volatile memory driver v1.3 [ 3.810064] Linux agpgart interface v0.103 [ 3.849803] virtio_blk virtio1: [vda] 133928 512-byte logical blocks (68.6 MB/65.4 MiB) [ 3.854112] vda: detected capacity change from 0 to 68571136 [ 3.871949] virtio_blk virtio2: [vdb] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) [ 3.875849] vdb: detected capacity change from 0 to 1073741824 [ 3.893826] virtio_blk virtio3: [vdc] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 3.897772] vdc: detected capacity change from 0 to 2621440000 [ 3.913667] virtio_blk virtio4: [vdd] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 3.917108] vdd: detected capacity change from 0 to 2621440000 [ 3.933802] virtio_blk virtio5: [vde] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 3.936835] vde: detected capacity change from 0 to 4294967296 [ 3.952586] virtio_blk virtio6: [vdf] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 3.956291] vdf: detected capacity change from 0 to 4294967296 [ 3.965811] libphy: Fixed MDIO Bus: probed [ 3.979972] usbcore: registered new interface driver usbserial_generic [ 3.988668] usbserial: USB Serial support registered for generic [ 3.991305] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12 [ 3.994735] serio: i8042 KBD port at 0x60,0x64 irq 1 [ 3.996330] serio: i8042 AUX port at 0x60,0x64 irq 12 [ 3.999048] mousedev: PS/2 mouse device common for all mice [ 4.002347] rtc_cmos 00:05: RTC can wake from S4 [ 4.005269] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input1 [ 4.005530] rtc_cmos 00:05: registered as rtc0 [ 4.011232] rtc_cmos 00:05: alarms up to one day, y3k, 242 bytes nvram, hpet irqs [ 4.013434] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input4 [ 4.014310] intel_pstate: CPU model not supported [ 4.018536] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input3 [ 4.023398] hid: raw HID events driver (C) Jiri Kosina [ 4.025247] usbcore: registered new interface driver usbhid [ 4.028113] usbhid: USB HID core driver [ 4.029690] drop_monitor: Initializing network drop monitor service [ 4.031736] Initializing XFRM netlink socket [ 4.033079] NET: Registered protocol family 10 [ 4.035720] Segment Routing with IPv6 [ 4.037588] NET: Registered protocol family 17 [ 4.040143] mpls_gso: MPLS GSO support [ 4.046167] RAS: Correctable Errors collector initialized. [ 4.048618] AVX version of gcm_enc/dec engaged. [ 4.050083] AES CTR mode by8 optimization enabled [ 4.163131] sched_clock: Marking stable (4163083318, 0)->(4900101609, -737018291) [ 4.167135] registered taskstats version 1 [ 4.169818] Loading compiled-in X.509 certificates [ 4.172429] zswap: loaded using pool lzo/zbud [ 4.206047] Key type big_key registered [ 4.237660] Key type encrypted registered [ 4.241262] ima: No TPM chip found, activating TPM-bypass! [ 4.243401] ima: Allocated hash algorithm: sha1 [ 4.245233] ima: No architecture policies found [ 4.248095] evm: Initialising EVM extended attributes: [ 4.250316] evm: security.selinux [ 4.251574] evm: security.ima [ 4.252800] evm: security.capability [ 4.254368] evm: HMAC attrs: 0x1 [ 4.258611] rtc_cmos 00:05: setting system clock to 2025-12-03 06:49:43 UTC (1764744583) [ 4.265727] debug: unmapping init [mem 0xffffffffa1203000-0xffffffffa13fffff] [ 4.269226] debug: unmapping init [mem 0xffffffff9ff82000-0xffffffffa0258fff] [ 4.278191] Write protecting the kernel read-only data: 28672k [ 4.281536] debug: unmapping init [mem 0xffffffff9e603000-0xffffffff9e7fffff] [ 4.284159] debug: unmapping init [mem 0xffffffff9ef14000-0xffffffff9effffff] [ 4.325698] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 4.334897] systemd[1]: Detected virtualization kvm. [ 4.336876] systemd[1]: Detected architecture x86-64. [ 4.338739] systemd[1]: Running in initial RAM disk. Welcome to Rocky Linux 8.10 (Green Obsidian) dracut-049-233.git20240115.el8 (Initramfs)! [ 4.366828] systemd[1]: No hostname configured. [ 4.368447] systemd[1]: Set hostname to . [ 4.370486] random: systemd: uninitialized urandom read (16 bytes read) [ 4.373064] systemd[1]: Initializing machine ID from random generator. [ 4.428107] random: ln: uninitialized urandom read (6 bytes read) [ 4.535337] random: systemd: uninitialized urandom read (16 bytes read) [ 4.538310] systemd[1]: Listening on udev Control Socket. [ OK ] Listening on udev Control Socket. [ 4.542972] systemd[1]: Listening on Journal Socket (/dev/log). [ OK ] Listening on Journal Socket (/dev/log). [ 4.551390] systemd[1]: Reached target Slices. [ OK ] Reached target Slices. [ OK ] Listening on udev Kernel Socket. [ OK ] Listening on Journal Socket. Starting Create list of required st…ce nodes for the current kernel... [ OK ] Reached target Sockets. Starting Setup Virtual Console... [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Reached target Local Encrypted Volumes. [ OK ] Reached target Local File Systems. Starting Journal Service... [ OK ] Reached target Paths. [ OK ] Reached target Timers. [ OK ] Reached target Initrd Root Device. Starting Apply Kernel Variables... Starting Create Volatile Files and Directories... [ OK ] Started Memstrack Anylazing Service. [ OK ] Reached target Swap. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Started Setup Virtual Console. [ OK ] Started Apply Kernel Variables. [ OK ] Started Journal Service. [ OK ] Started Create Volatile Files and Directories. Starting dracut cmdline hook... Starting Create Static Device Nodes in /dev... [ OK ] Started Create Static Device Nodes in /dev. [ OK ] Started dracut cmdline hook. Starting dracut pre-udev hook... [ 5.482235] device-mapper: uevent: version 1.0.3 [ 5.484540] device-mapper: ioctl: 4.46.0-ioctl (2022-02-22) initialised: dm-devel@redhat.com [ OK ] Started dracut pre-udev hook. Starting udev Kernel Device Manager... [ OK ] Started udev Kernel Device Manager. Starting dracut pre-trigger hook... [ OK ] Started dracut pre-trigger hook. Starting udev Coldplug all Devices... Mounting Kernel Configuration File System... [ OK ] Mounted Kernel Configuration File System. [ OK ] Started udev Coldplug all Devices. [ OK ] Reached target System Initialization. [ OK ] Reached target Basic System. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. [ 6.537096] random: fast init done Starting dracut [ 6.542949] virtio_net virtio0 ens2: renamed from eth0 initqueue hook... [ 6.872670] scsi host0: ata_piix [ 7.183359] scsi host1: ata_piix [ 7.184588] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc320 irq 14 [ 7.186346] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc328 irq 15 [ 11.773445] random: crng init done [ 11.774597] random: 7 urandom warning(s) missed due to ratelimiting [ 13.427547] hrtimer: interrupt took 34088011 ns [ 14.325343] dracut-initqueue[594]: RTNETLINK answers: File exists Starting nbd nbd0... [ OK ] Started nbd nbd0. [ OK ] Started dracut initqueue hook. Mounting /sysroot... [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. [ 16.066059] EXT4-fs (nbd0): mounted filesystem with ordered data mode. Opts: (null) [ OK ] Mounted /sysroot. [ OK ] Reached target Initrd Root File System. Starting Reload Configuration from the Real Root... [ OK ] Started Reload Configuration from the Real Root. [ OK ] Reached target Initrd File Systems. [ OK ] Reached target Initrd Default Target. Starting dracut pre-pivot and cleanup hook... [ OK ] Started dracut pre-pivot and cleanup hook. Starting Cleaning Up and Shutting Down Daemons... [ OK ] Stopped dracut pre-pivot and cleanup hook. [ OK ] Stopped target Timers. [ OK ] Stopped target Initrd Default Target. [ OK ] Stopped target Initrd Root Device. [ OK ] Stopped target Remote File Systems. [ OK ] Stopped target Remote File Systems (Pre). [ OK ] Stopped dracut initqueue hook. Stopping Hardware RNG Entropy Gatherer Daemon... [ OK ] Stopped Hardware RNG Entropy Gatherer Daemon. [ OK ] Stopped target Basic System. [ OK ] Stopped target Sockets. [ OK ] Stopped target Slices. [ OK ] Stopped target Paths. [ OK ] Stopped target System Initialization. [ OK ] Stopped udev Coldplug all Devices. [ OK ] Stopped dracut pre-trigger hook. Stopping udev Kernel Device Manager... [ OK ] Stopped Apply Kernel Variables. [ OK ] Stopped target Local Encrypted Volumes. [ OK ] Stopped Dispatch Password Requests to Console Directory Watch. [ OK ] Stopped target Swap. [ OK ] Stopped Create Volatile Files and Directories. [ OK ] Stopped target Local File Systems. [ OK ] Stopped udev Kernel Device Manager. [ OK ] Started Cleaning Up and Shutting Down Daemons. [ OK ] Stopped Create Static Device Nodes in /dev. [ OK ] Stopped Create list of required sta…vice nodes for the current kernel. [ OK ] Stopped dracut pre-udev hook. [ OK ] Stopped dracut cmdline hook. [ OK ] Closed udev Control Socket. [ OK ] Closed udev Kernel Socket. Starting Cleanup udevd DB... [ OK ] Started Cleanup udevd DB. [ OK ] Reached target Switch Root. Starting Switch Root... [ 17.650497] printk: systemd: 21 output lines suppressed due to ratelimiting [ 18.165664] SELinux: Disabled at runtime. [ 18.239099] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 18.251607] systemd[1]: Detected virtualization kvm. [ 18.253696] systemd[1]: Detected architecture x86-64. Welcome to Rocky Linux 8.10 (Green Obsidian)! [ 19.190995] systemd[1]: initrd-switch-root.service: Succeeded. [ 19.197768] systemd[1]: Stopped Switch Root. [ OK ] Stopped Switch Root. [ 19.208028] systemd[1]: systemd-journald.service: Service has no hold-off time (RestartSec=0), scheduling restart. [ 19.220087] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. [ 19.226100] systemd[1]: Stopped Journal Service. [ OK ] Stopped Journal Service. [ 19.254395] systemd[1]: Starting Journal Service... Starting Journal Service... [ 19.270026] systemd[1]: Stopped target Switch Root. [ OK ] Stopped target Switch Root. [ OK ] Stopped target Initrd File Systems. [ OK ] Listening on initctl Compatibility Named Pipe. [ OK ] Created slice User and Session Slice. [ OK ] Started Forward Password Requests to Wall Directory Watch. Starting Remount Root and Kernel File Systems... Starting Apply Kernel Variables... Mounting Kernel Debug File System... Activating swap /dev/disk/by-label/SWAP... [ OK ] Reached target rpc_pipefs.target. [ OK ] Listening on udev Control Socket. [ OK ] Listening on RPCbind Server Activation Socket. [ 19.548711] Adding 1048572k swap on /dev/vdb. Priority:-2 extents:1 across:1048572k FS [ OK ] Listening on Process Core Dump Socket. [ OK ] Reached target Slices. Starting Create list of required st…ce nodes for the current kernel... [FAILED] Failed to set up automount Arbitrar…rmats File System Automount Point. See 'systemctl status proc-sys-fs-binfmt_misc.automount' for details. [ OK ] Created slice system-sshd\x2dkeygen.slice. Mounting Huge Pages File System... [ OK ] Created slice system-serial\x2dgetty.slice. [ OK ] Created slice system-getty.slice. [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Reached target Local Encrypted Volumes. Mounting POSIX Message Queue File System... [ OK ] Stopped target Initrd Root File System. [ OK ] Reached target RPC Port Mapper. [ OK ] Listening on udev Kernel Socket. Starting udev Coldplug all Devices... [ OK ] Reached target Paths. [ OK ] Started Journal Service. [FAILED] Failed to start Remount Root and Kernel File Systems. See 'systemctl status systemd-remount-fs.service' for details. [ OK ] Started Apply Kernel Variables. [ OK ] Mounted Kernel Debug File System. [ OK ] Activated swap /dev/disk/by-label/SWAP. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Mounted Huge Pages File System. [ OK ] Mounted POSIX Message Queue File System. [ OK ] Reached target Swap. Starting Create Static Device Nodes in /dev... Starting Configure read-only root support... Starting Flush Journal to Persistent Storage... [ OK ] Started Flush Journal to Persistent Storage. [ OK ] Started Create Static Device Nodes in /dev. [ OK ] Reached target Local File Systems (Pre). Mounting /home/green/git/lustre-release... Mounting /mnt... Starting udev Kernel Device Manager... [ OK ] Started udev Coldplug all Devices. [ OK ] Mounted /mnt. [ 20.312389] squashfs: version 4.0 (2009/01/31) Phillip Lougher [ OK ] Mounted /home/green/git/lustre-release. [ OK ] Started udev Kernel Device Manager. [ 20.961600] piix4_smbus 0000:00:01.3: SMBus Host Controller at 0x700, revision 0 [ 21.052522] input: PC Speaker as /devices/platform/pcspkr/input/input5 [ 21.335854] RAPL PMU: API unit is 2^-32 Joules, 0 fixed counters, 10737418240 ms ovfl timer [ 21.421343] EDAC sbridge: Ver: 1.1.2 [ 24.619942] Key type dns_resolver registered [ 24.998844] NFS: Registering the id_resolver key type [ 25.000932] Key type id_resolver registered [ 25.002375] Key type id_legacy registered [ OK ] Started Configure read-only root support. Starting Load/Save Random Seed... [ OK ] Reached target Local File Systems. Starting Mark the need to relabel after reboot... Starting Rebuild Dynamic Linker Cache... Starting Create Volatile Files and Directories... [ OK ] Started Load/Save Random Seed. [ OK ] Started Mark the need to relabel after reboot. [ OK ] Started Create Volatile Files and Directories. Starting Update UTMP about System Boot/Shutdown... Starting RPC Bind... [ OK ] Started Update UTMP about System Boot/Shutdown. [ OK ] Started RPC Bind. [ OK ] Started Rebuild Dynamic Linker Cache. Starting Update is Completed... [ OK ] Started Update is Completed. [ OK ] Reached target System Initialization. [ OK ] Started Daily Cleanup of Temporary Directories. [ OK ] Listening on D-Bus System Message Bus Socket. [ OK ] Reached target Sockets. [ OK ] Started dnf makecache --timer. [ OK ] Reached target Basic System. Starting Restore /run/initramfs on shutdown... [ OK ] Started Hardware RNG Entropy Gatherer Daemon. [ OK ] Started D-Bus System Message Bus. Starting Network Manager... [ OK ] Reached target sshd-keygen.target. [ OK ] Started irqbalance daemon. Starting Login Service... [ OK ] Started daily update of the root trust anchor for DNSSEC. [ OK ] Reached target Timers. [ OK ] Started Restore /run/initramfs on shutdown. [ OK ] Started Network Manager. [ OK ] Reached target Network. Starting GSSAPI Proxy Daemon... Starting Dynamic System Tuning Daemon... Starting OpenSSH server daemon... Starting Network Manager Wait Online... [ OK ] Started GSSAPI Proxy Daemon. [ OK ] Reached target NFS client services. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Starting Permit User Sessions... [ OK ] Started Permit User Sessions. [ OK ] Started OpenSSH server daemon. [ OK ] Started Serial Getty on ttyS0. [ OK ] Started Getty on tty1. [ OK ] Started Command Scheduler. [ OK ] Started Serial Getty on ttyS1. [ OK ] Reached target Login Prompts. Starting Hostname Service... [ OK ] Started Login Service. [ OK ] Started Hostname Service. Starting Network Manager Script Dispatcher Service... [ OK ] Started Network Manager Script Dispatcher Service. [ OK ] Started Network Manager Wait Online. [ OK ] Reached target Network is Online. Starting Crash recovery kernel arming... Starting System Logging Service... Starting Notify NFS peers of a restart... [ OK ] Started Notify NFS peers of a restart. [ OK ] Started System Logging Service. Starting Authorization Manager... Rocky Linux 8.10 (Green Obsidian) Kernel 4.18.0rh8.10-debug on an x86_64 oleg118-server login: [ 53.536174] libcfs: loading out-of-tree module taints kernel. [ 53.554680] Key type ._llcrypt registered [ 53.556177] Key type .llcrypt registered [ 53.597671] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_hostid [ 60.942433] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing load_modules_local [ 61.583887] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 61.590510] alg: No test for adler32 (adler32-zlib) [ 62.578163] Lustre: Lustre: Build Version: 2.17.0_RC1_2_g29d2772 [ 62.867479] LNet: Added LNI 192.168.201.118@tcp [8/256/0/180] [ 64.479220] Key type lgssc registered [ 65.114730] Lustre: Echo OBD driver; http://www.lustre.org/ [ 71.782498] ZFS: Loaded module v2.3.2-1, ZFS pool version 5000, ZFS filesystem version 5 [ 72.515108] LDISKFS-fs (vdc): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 76.147040] LDISKFS-fs (vdd): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 78.862782] LDISKFS-fs (vde): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 81.698758] LDISKFS-fs (vdf): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 87.593431] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing load_modules_local [ 92.252241] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 92.276572] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 92.283524] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 93.387623] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 93.402429] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 93.446159] Lustre: lustre-MDT0000: new disk, initializing [ 93.477689] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 93.487206] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 94.916982] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 100.137419] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 100.170527] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 100.202035] Lustre: 6491:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 100.217373] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 100.220163] Lustre: Skipped 1 previous similar message [ 100.263370] Lustre: lustre-MDT0001: new disk, initializing [ 100.286349] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 100.296768] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 100.300537] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 101.640121] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 104.035086] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 107.509990] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 107.539288] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 107.636852] Lustre: lustre-OST0000: new disk, initializing [ 107.638537] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 107.656400] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 109.736027] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 113.129549] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 113.132313] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 113.147049] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 115.413096] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 115.441758] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 115.479034] Lustre: lustre-OST0001: new disk, initializing [ 115.481421] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 115.501364] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 117.436254] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 118.186755] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 118.190373] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 118.205787] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 123.456324] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 130.298355] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 137.354678] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing check_logdir /tmp/testlogs/ [ 139.050868] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing yml_node [ 141.160064] Lustre: DEBUG MARKER: Client: 2.17.0.RC1 [ 142.174335] Lustre: DEBUG MARKER: MDS: 2.17.0.RC1 [ 143.159112] Lustre: DEBUG MARKER: OSS: 2.17.0.RC1 [ 143.807353] Lustre: DEBUG MARKER: -----============= acceptance-small: conf-sanity ============----- Wed Dec 3 01:52:02 EST 2025 [ 151.474835] Lustre: DEBUG MARKER: excepting tests: 102 106 115 32newtarball [ 152.177334] Lustre: DEBUG MARKER: skipping tests SLOW=no: 45 69 106 111 114 [ 164.320775] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 164.321528] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 164.327233] Lustre: Skipped 2 previous similar messages [ 169.440898] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 169.444128] Lustre: Skipped 6 previous similar messages [ 170.445492] LustreError: 12193:0:(obd_class.h:479:obd_check_dev()) Device 28 not setup [ 170.487831] Lustre: server umount lustre-MDT0000 complete [ 172.189272] LustreError: 6483:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1764744751 with bad export cookie 3819214302548825768 [ 172.191123] LustreError: MGC192.168.201.118@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 172.193210] LustreError: 6483:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 174.559827] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 174.559996] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 174.565538] Lustre: Skipped 2 previous similar messages [ 177.568659] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 177.571128] Lustre: Skipped 2 previous similar messages [ 178.313364] LustreError: 12395:0:(obd_class.h:479:obd_check_dev()) Device 27 not setup [ 178.316496] LustreError: 12395:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 178.372454] Lustre: server umount lustre-MDT0001 complete [ 180.174247] LustreError: 12597:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 180.177603] LustreError: 12597:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 180.210848] Lustre: server umount lustre-OST0000 complete [ 181.995256] Lustre: server umount lustre-OST0001 complete [ 184.910719] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_hostid [ 189.272497] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing load_modules_local [ 193.790843] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 196.836599] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 199.709629] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 202.769215] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 206.148849] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 206.189849] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 206.311421] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 206.332946] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 206.378514] Lustre: lustre-MDT0000: new disk, initializing [ 206.428906] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 206.444451] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 208.381120] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 212.924309] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 212.952171] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 212.986852] Lustre: 14979:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 213.098893] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 214.488845] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 217.675762] Lustre: DEBUG MARKER: oleg118-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 218.890539] Lustre: DEBUG MARKER: oleg118-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 221.143787] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 221.180701] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 221.289423] Lustre: lustre-OST0000: new disk, initializing [ 221.291489] Lustre: Skipped 1 previous similar message [ 221.293971] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 221.296882] Lustre: Skipped 2 previous similar messages [ 221.331189] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 221.334447] Lustre: Skipped 1 previous similar message [ 222.521097] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 222.524509] Lustre: Skipped 1 previous similar message [ 222.526657] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 222.563496] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 223.314587] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 226.516976] Lustre: DEBUG MARKER: oleg118-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 227.935620] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 228.017927] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 229.380829] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 229.451782] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 232.928807] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 232.930171] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 232.932307] LustreError: Skipped 1 previous similar message [ 232.937258] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 232.940450] Lustre: Skipped 1 previous similar message [ 236.141187] LustreError: 17915:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 236.144156] LustreError: 17915:0:(obd_class.h:479:obd_check_dev()) Skipped 9 previous similar messages [ 236.180163] Lustre: server umount lustre-OST0000 complete [ 238.048669] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 238.052481] Lustre: Skipped 2 previous similar messages [ 243.168351] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 243.171420] Lustre: Skipped 3 previous similar messages [ 243.518692] LustreError: 18117:0:(obd_class.h:479:obd_check_dev()) Device 22 not setup [ 243.520674] LustreError: 18117:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 243.549155] Lustre: server umount lustre-MDT0000 complete [ 244.765114] LustreError: 14971:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1764744824 with bad export cookie 3819214302548827308 [ 244.768194] LustreError: MGC192.168.201.118@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 244.769754] LustreError: 14971:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 244.893078] Lustre: server umount lustre-MDT0001 complete [ 246.531024] Lustre: DEBUG MARKER: == conf-sanity test 76a: set permanent params with lctl across mounts ========================================================== 01:53:45 (1764744825) [ 248.675430] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 248.827101] LustreError: 18845:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 248.833496] LustreError: 18845:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 5 previous similar messages [ 248.851415] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 250.113846] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 252.974633] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 254.438592] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 256.350129] Lustre: DEBUG MARKER: oleg118-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 257.684446] Lustre: DEBUG MARKER: oleg118-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 259.974558] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 261.979357] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 263.695511] Lustre: DEBUG MARKER: oleg118-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 280.979043] Lustre: Modifying parameter general.osc.*.max_dirty_mb=452 in log params [ 284.128052] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 284.130886] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 284.136175] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 290.106190] LustreError: 21168:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup [ 290.108277] LustreError: 21168:0:(obd_class.h:479:obd_check_dev()) Skipped 13 previous similar messages [ 290.134923] Lustre: server umount lustre-MDT0000 complete [ 291.339103] LustreError: 18824:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1764744870 with bad export cookie 3819214302548828190 [ 291.342245] LustreError: MGC192.168.201.118@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 291.343588] LustreError: 18824:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 291.453173] Lustre: server umount lustre-MDT0001 complete [ 303.059855] Lustre: server umount lustre-OST0000 complete [ 309.514664] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing load_modules_local [ 313.143488] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 313.280498] LustreError: 22895:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 313.296340] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 313.298533] Lustre: Skipped 2 previous similar messages [ 314.719094] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 317.622521] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 319.128717] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 320.182241] Lustre: Setting parameter general.debug_raw_pointers=Y in log params [ 322.506666] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 324.511377] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 327.440505] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 327.463420] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 327.526646] Lustre: lustre-OST0001: new disk, initializing [ 327.529324] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 328.588073] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 328.593363] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 328.643923] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 329.520878] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 333.997944] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 341.647540] Lustre: Setting parameter general.obdfilter.*.client_cache_count=256 in log params [ 341.649552] Lustre: Skipped 1 previous similar message [ 348.640086] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 348.643941] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 348.648981] Lustre: Skipped 2 previous similar messages [ 348.651153] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 348.653065] Lustre: Skipped 5 previous similar messages [ 350.722253] LustreError: 26374:0:(obd_class.h:479:obd_check_dev()) Device 28 not setup [ 350.724875] LustreError: 26374:0:(obd_class.h:479:obd_check_dev()) Skipped 17 previous similar messages [ 350.764588] Lustre: server umount lustre-MDT0000 complete [ 352.091671] LustreError: 22875:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1764744931 with bad export cookie 3819214302548829037 [ 352.095453] LustreError: MGC192.168.201.118@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 352.096573] LustreError: 22875:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 352.212202] Lustre: server umount lustre-MDT0001 complete [ 363.456970] Lustre: server umount lustre-OST0000 complete [ 375.124715] Lustre: server umount lustre-OST0001 complete [ 380.165483] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing load_modules_local [ 383.527094] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 383.678975] LustreError: 28149:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 383.700577] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 383.702698] Lustre: Skipped 3 previous similar messages [ 384.990509] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 387.911937] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 389.352396] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 390.259742] Lustre: 29265:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 392.303656] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 393.442484] LustreError: 29621:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 394.212527] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 396.970969] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 398.915399] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 401.977211] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 403.098800] Lustre: 31117:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 413.670495] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 413.671680] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 413.673058] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 413.673068] Lustre: Skipped 4 previous similar messages [ 413.675542] Lustre: Skipped 6 previous similar messages [ 415.739761] LustreError: 31320:0:(obd_class.h:479:obd_check_dev()) Device 10 not setup [ 415.741673] LustreError: 31320:0:(obd_class.h:479:obd_check_dev()) Skipped 25 previous similar messages [ 415.767108] Lustre: server umount lustre-MDT0000 complete [ 416.958044] LustreError: 28883:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1764744996 with bad export cookie 3819214302548830332 [ 416.959823] LustreError: MGC192.168.201.118@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 416.961748] LustreError: 28883:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 417.064844] Lustre: server umount lustre-MDT0001 complete [ 427.972319] Lustre: server umount lustre-OST0000 complete [ 439.590453] Lustre: server umount lustre-OST0001 complete [ 442.431965] Lustre: DEBUG MARKER: == conf-sanity test 76b: verify params log setup correctly ========================================================== 01:57:01 (1764745021) [ 451.033021] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing load_modules_local [ 454.255301] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 454.378649] LustreError: 33898:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 455.653151] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 458.391991] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 459.808404] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 460.825351] Lustre: 35014:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 463.077587] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 464.226943] LustreError: 35370:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 465.158522] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 468.140077] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 470.260812] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 473.358800] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 474.524417] Lustre: 36868:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 484.319990] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 484.323081] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 486.844131] LustreError: 37021:0:(obd_class.h:479:obd_check_dev()) Device 10 not setup [ 486.846083] LustreError: 37021:0:(obd_class.h:479:obd_check_dev()) Skipped 25 previous similar messages [ 486.872323] Lustre: server umount lustre-MDT0000 complete [ 488.060459] LustreError: 33878:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1764745067 with bad export cookie 3819214302548831438 [ 488.065388] LustreError: MGC192.168.201.118@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 488.179354] Lustre: server umount lustre-MDT0001 complete [ 499.137213] Lustre: server umount lustre-OST0000 complete [ 510.547175] Lustre: server umount lustre-OST0001 complete [ 513.400859] Lustre: DEBUG MARKER: == conf-sanity test 76c: verify changelog_mask is applied with lctl set_param -P ========================================================== 01:58:12 (1764745092) [ 518.719865] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing load_modules_local [ 522.349670] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 522.490990] LustreError: 38991:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 522.514495] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 522.516569] Lustre: Skipped 7 previous similar messages [ 523.954618] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 527.029644] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 528.529458] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 529.595758] Lustre: 40109:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 531.776488] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 533.905249] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 537.023824] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 539.103079] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 542.328127] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 548.932931] Lustre: Setting parameter general.mdd.*.changelog_mask=-CLOSE in log params [ 550.028215] Lustre: lustre-MDT0000: Not available for connect from 192.168.201.18@tcp (stopping) [ 550.030230] Lustre: Skipped 7 previous similar messages [ 552.417424] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 552.423455] Lustre: Skipped 5 previous similar messages [ 556.119378] Lustre: server umount lustre-MDT0000 complete [ 557.355079] LustreError: 41962:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1764745136 with bad export cookie 3819214302548832544 [ 557.358112] LustreError: MGC192.168.201.118@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 557.359568] LustreError: 41962:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 6 previous similar messages [ 557.536035] LustreError: 39020:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 557.542477] LustreError: 39020:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 4 previous similar messages [ 563.571695] Lustre: server umount lustre-MDT0001 complete [ 566.534775] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 567.647137] LustreError: 42821:0:(import.c:337:ptlrpc_invalidate_import()) MGS: timeout waiting for callback (1 != 0) [ 567.649864] LustreError: 42821:0:(import.c:361:ptlrpc_invalidate_import()) @@@ still on sending list req@ffff897581d77b80 x1850468878032000/t0(0) o250->MGC192.168.201.118@tcp@0@lo:26/25 lens 520/544 e 0 to 0 dl 1764745146 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'ptlrpcd_00_01.0' uid:0 gid:0 projid:4294967295 [ 567.658480] LustreError: 42821:0:(import.c:371:ptlrpc_invalidate_import()) MGS: Unregistering RPCs found (0). Network is sluggish? Waiting for them to error out. [ 567.778071] Lustre: Evicted from MGS (at 0@lo) after server handle changed from 0x0 to 0x3500932c5083b595 [ 567.782176] Lustre: MGC192.168.201.118@tcp: Connection restored to 0@lo (at 0@lo) [ 569.319650] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 572.419268] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 573.967476] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 575.732666] Lustre: DEBUG MARKER: oleg118-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 588.262538] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 590.705568] Lustre: DEBUG MARKER: mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 14 sec [ 591.848213] Lustre: DEBUG MARKER: oleg118-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 592.389444] Lustre: DEBUG MARKER: mdc.lustre-MDT0001-mdc-*.mds_server_uuid in FULL state after 0 sec [ 593.382718] Lustre: lustre-MDT0000-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 593.384815] Lustre: Skipped 1 previous similar message [ 598.498329] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 599.771201] Lustre: server umount lustre-MDT0000 complete [ 601.024095] LustreError: 38971:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1764745180 with bad export cookie 3819214302548833685 [ 601.026654] LustreError: MGC192.168.201.118@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 601.028051] LustreError: 38971:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 601.141373] Lustre: server umount lustre-MDT0001 complete [ 612.806883] Lustre: server umount lustre-OST0000 complete [ 624.371816] LustreError: 44980:0:(obd_class.h:479:obd_check_dev()) Device 2 not setup [ 624.374698] LustreError: 44980:0:(obd_class.h:479:obd_check_dev()) Skipped 61 previous similar messages [ 624.422624] Lustre: server umount lustre-OST0001 complete [ 627.488721] Lustre: DEBUG MARKER: == conf-sanity test 76d: verify llite.*.xattr_cache can be set by 'lctl set_param -P' correctly ========================================================== 02:00:06 (1764745206) [ 632.768082] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing load_modules_local [ 636.236547] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 636.376077] LustreError: 46347:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 636.385528] LustreError: 46347:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 4 previous similar messages [ 637.713779] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 640.689390] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 642.235160] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 643.249868] Lustre: 47464:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 643.253385] Lustre: 47464:0:(mgs_llog.c:1346:mgs_modify_param()) Skipped 1 previous similar message [ 645.444508] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 647.388232] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 650.193191] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 652.078388] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 655.094304] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 661.526060] Lustre: Setting parameter general.llite.*.xattr_cache=0 in log params [ 666.593846] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 671.842585] Lustre: server umount lustre-MDT0000 complete [ 673.173224] LustreError: 47082:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1764745252 with bad export cookie 3819214302548834497 [ 673.177758] LustreError: MGC192.168.201.118@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 673.178518] LustreError: 47082:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 673.298421] Lustre: server umount lustre-MDT0001 complete [ 684.043262] Lustre: server umount lustre-OST0000 complete [ 695.515067] Lustre: server umount lustre-OST0001 complete [ 698.607490] Lustre: DEBUG MARKER: == conf-sanity test 77: comma-separated MGS NIDs and failover node NIDs ========================================================== 02:01:17 (1764745277) [ 699.441114] Lustre: DEBUG MARKER: SKIP: conf-sanity test_77 mixed loopback and real device not working [ 700.024314] Lustre: DEBUG MARKER: == conf-sanity test 78: run resize2fs on MDT and OST filesystems ========================================================== 02:01:18 (1764745278) [ 703.349589] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_hostid [ 705.925572] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing load_modules_local [ 709.072512] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 711.026292] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 713.314653] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 713.336744] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 713.424745] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 713.462464] Lustre: lustre-MDT0000: new disk, initializing [ 713.494538] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 714.812738] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 717.924311] Lustre: DEBUG MARKER: oleg118-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 720.092574] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 720.115103] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 721.974658] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 722.175555] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 722.184981] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 724.995109] Lustre: DEBUG MARKER: oleg118-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 725.636000] Lustre: DEBUG MARKER: create test files [ 737.760903] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 737.761130] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 737.767006] Lustre: Skipped 11 previous similar messages [ 741.057830] Lustre: server umount lustre-OST0000 complete [ 742.390658] Lustre: server umount lustre-MDT0000 complete [ 746.028242] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing unload_modules_local [ 747.144225] Key type lgssc unregistered [ 747.275449] LNet: 54862:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 747.278437] LNetError: 54862:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 747.285360] LNet: Removed LNI 192.168.201.118@tcp [ 747.649118] Key type .llcrypt unregistered [ 747.650234] Key type ._llcrypt unregistered [ 758.635272] Key type ._llcrypt registered [ 758.636841] Key type .llcrypt registered [ 758.680748] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing load_modules_local [ 759.001554] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 759.010115] alg: No test for adler32 (adler32-zlib) [ 759.872292] Lustre: Lustre: Build Version: 2.17.0_RC1_2_g29d2772 [ 759.965385] LNet: Added LNI 192.168.201.118@tcp [8/256/0/180] [ 761.551126] Key type lgssc registered [ 761.964594] Lustre: Echo OBD driver; http://www.lustre.org/ [ 764.655828] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 764.665293] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 765.830672] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 767.173177] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 769.002164] Lustre: DEBUG MARKER: oleg118-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 771.425714] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 771.546706] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 773.542410] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 775.392159] Lustre: DEBUG MARKER: oleg118-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 776.676232] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:98 to 0x240000400:129) [ 776.701935] Lustre: 57443:0:(osd_internal.h:1459:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 515 < left 544, rollback = 9 [ 776.704647] Lustre: 57443:0:(osd_handler.c:2096:osd_trans_dump_creds()) create: 32/128/0, destroy: 0/0/0 [ 776.706666] Lustre: 57443:0:(osd_handler.c:2103:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 776.708824] Lustre: 57443:0:(osd_handler.c:2113:osd_trans_dump_creds()) write: 2/13/1, punch: 0/0/0, quota 1/3/0 [ 776.710907] Lustre: 57443:0:(osd_handler.c:2120:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 776.713018] Lustre: 57443:0:(osd_handler.c:2127:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 783.832244] Lustre: DEBUG MARKER: check files after expanding the MDT and OST filesystems [ 785.421723] Lustre: DEBUG MARKER: create more files after expanding the MDT and OST filesystems [ 792.032557] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 792.032903] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 792.038475] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 793.456097] LustreError: 58188:0:(obd_class.h:479:obd_check_dev()) Device 13 not setup [ 793.487414] Lustre: server umount lustre-OST0000 complete [ 794.752482] LustreError: 58390:0:(obd_class.h:479:obd_check_dev()) Device 8 not setup [ 794.754340] LustreError: 58390:0:(obd_class.h:479:obd_check_dev()) Skipped 1 previous similar message [ 794.810635] Lustre: server umount lustre-MDT0000 complete [ 798.528395] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing unload_modules_local [ 799.537584] Key type lgssc unregistered [ 799.670479] LNet: 58921:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 799.672961] LNetError: 58921:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 799.682298] LNet: Removed LNI 192.168.201.118@tcp [ 799.968267] Key type .llcrypt unregistered [ 799.969295] Key type ._llcrypt unregistered [ 811.348681] Key type ._llcrypt registered [ 811.349632] Key type .llcrypt registered [ 811.386670] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing load_modules_local [ 811.703228] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 811.756736] alg: No test for adler32 (adler32-zlib) [ 812.620031] Lustre: Lustre: Build Version: 2.17.0_RC1_2_g29d2772 [ 812.707347] LNet: Added LNI 192.168.201.118@tcp [8/256/0/180] [ 814.287162] Key type lgssc registered [ 814.658630] Lustre: Echo OBD driver; http://www.lustre.org/ [ 817.314405] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 817.320918] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 818.474350] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 819.771344] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 821.584136] Lustre: DEBUG MARKER: oleg118-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 824.056863] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 824.176725] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 826.295272] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 828.223523] Lustre: DEBUG MARKER: oleg118-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 828.263722] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:140 to 0x240000400:161) [ 828.290912] Lustre: 61601:0:(osd_internal.h:1459:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 515 < left 544, rollback = 9 [ 828.295826] Lustre: 61601:0:(osd_handler.c:2096:osd_trans_dump_creds()) create: 32/128/0, destroy: 0/0/0 [ 828.297777] Lustre: 61601:0:(osd_handler.c:2103:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 828.299657] Lustre: 61601:0:(osd_handler.c:2113:osd_trans_dump_creds()) write: 2/13/1, punch: 0/0/0, quota 1/3/0 [ 828.302863] Lustre: 61601:0:(osd_handler.c:2120:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 828.305928] Lustre: 61601:0:(osd_handler.c:2127:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 831.279746] Lustre: DEBUG MARKER: check files after shrinking the MDT and OST filesystems [ 833.504348] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 833.504673] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 833.510557] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 835.503924] LustreError: 62293:0:(obd_class.h:479:obd_check_dev()) Device 13 not setup [ 835.543418] Lustre: server umount lustre-OST0000 complete [ 842.943218] LustreError: 62495:0:(obd_class.h:479:obd_check_dev()) Device 8 not setup [ 842.945746] LustreError: 62495:0:(obd_class.h:479:obd_check_dev()) Skipped 1 previous similar message [ 843.002675] Lustre: server umount lustre-MDT0000 complete [ 846.295133] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing unload_modules_local [ 847.230515] Key type lgssc unregistered [ 847.353401] LNet: 63028:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 847.355299] LNetError: 63028:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 847.361389] LNet: Removed LNI 192.168.201.118@tcp [ 847.647053] Key type .llcrypt unregistered [ 847.647985] Key type ._llcrypt unregistered [ 853.291171] Key type ._llcrypt registered [ 853.292137] Key type .llcrypt registered [ 853.327635] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_hostid [ 859.529937] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing load_modules_local [ 859.835422] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 859.846499] alg: No test for adler32 (adler32-zlib) [ 860.700395] Lustre: Lustre: Build Version: 2.17.0_RC1_2_g29d2772 [ 860.791951] LNet: Added LNI 192.168.201.118@tcp [8/256/0/180] [ 862.375193] Key type lgssc registered [ 862.771805] Lustre: Echo OBD driver; http://www.lustre.org/ [ 865.791276] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 868.049941] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 870.206544] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 872.312573] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 874.592697] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 874.615225] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 874.620626] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 875.703268] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 875.716102] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 875.751322] Lustre: lustre-MDT0000: new disk, initializing [ 875.775960] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 875.782453] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 877.035477] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 881.087620] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 881.107149] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 881.138605] Lustre: 66127:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 881.159574] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 881.161688] Lustre: Skipped 1 previous similar message [ 881.196509] Lustre: lustre-MDT0001: new disk, initializing [ 881.227933] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 881.239336] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 881.243351] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 882.470790] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 885.519521] Lustre: DEBUG MARKER: oleg118-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 886.687649] Lustre: DEBUG MARKER: oleg118-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 888.953765] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 888.976651] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 889.070115] Lustre: lustre-OST0000: new disk, initializing [ 889.072427] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 889.099674] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 891.003604] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 894.015548] Lustre: DEBUG MARKER: oleg118-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 895.332984] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 899.063308] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 899.067162] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 899.106555] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 899.483247] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 4 sec [ 900.925422] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 901.007245] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 904.161527] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 904.162108] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 904.164438] LustreError: Skipped 1 previous similar message [ 904.170657] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 904.174871] Lustre: Skipped 1 previous similar message [ 907.820120] LustreError: 69096:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 907.863268] Lustre: server umount lustre-OST0000 complete [ 909.280514] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 909.285590] Lustre: Skipped 1 previous similar message [ 909.289460] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 909.292121] Lustre: Skipped 1 previous similar message [ 914.400105] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 914.402468] Lustre: Skipped 1 previous similar message [ 915.327771] LustreError: 69297:0:(obd_class.h:479:obd_check_dev()) Device 22 not setup [ 915.330726] LustreError: 69297:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 915.359794] Lustre: server umount lustre-MDT0000 complete [ 916.629766] LustreError: 66119:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1764745495 with bad export cookie 9843931598298569755 [ 916.633416] LustreError: MGC192.168.201.118@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 916.634469] LustreError: 66119:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 916.699161] LustreError: 69498:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 916.701436] LustreError: 69498:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 916.752425] Lustre: server umount lustre-MDT0001 complete [ 919.544322] Lustre: DEBUG MARKER: == conf-sanity test 79: format MDT/OST without mgs option (should return errors) ========================================================== 02:04:58 (1764745498) [ 922.814592] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing load_modules_local [ 927.074394] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 935.563064] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_hostid [ 938.786406] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing load_modules_local [ 941.909846] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 944.040876] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 945.989939] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 948.133099] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 950.397937] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 950.420627] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 950.504994] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 950.521562] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 950.558735] Lustre: lustre-MDT0000: new disk, initializing [ 950.584110] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 950.591126] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 951.925960] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 956.215887] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 956.234463] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 956.270705] Lustre: 73727:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 956.293779] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 956.296082] Lustre: Skipped 1 previous similar message [ 956.333484] Lustre: lustre-MDT0001: new disk, initializing [ 956.370405] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 956.383334] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 956.387803] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 957.778749] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 960.969179] Lustre: DEBUG MARKER: oleg118-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 962.232996] Lustre: DEBUG MARKER: oleg118-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 964.707562] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 964.735575] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 964.830473] Lustre: lustre-OST0000: new disk, initializing [ 964.832793] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 964.862493] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 966.910270] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 970.019338] Lustre: DEBUG MARKER: oleg118-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 970.228933] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 970.232408] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 970.259948] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 971.413336] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 971.494182] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 972.954720] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 973.031753] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 975.328881] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 975.330261] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 975.331647] LustreError: Skipped 1 previous similar message [ 975.331731] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 975.342851] Lustre: Skipped 2 previous similar messages [ 979.820845] LustreError: 76667:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 979.822759] LustreError: 76667:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 979.858430] Lustre: server umount lustre-OST0000 complete [ 981.983794] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 981.985965] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 981.989652] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 981.991378] Lustre: Skipped 1 previous similar message [ 987.198880] LustreError: 76868:0:(obd_class.h:479:obd_check_dev()) Device 22 not setup [ 987.200712] LustreError: 76868:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 987.229189] Lustre: server umount lustre-MDT0000 complete [ 988.464177] LustreError: 73718:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1764745567 with bad export cookie 9843931598298570623 [ 988.466585] LustreError: MGC192.168.201.118@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 988.467906] LustreError: 73718:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 988.569742] Lustre: server umount lustre-MDT0001 complete [ 991.229051] Lustre: DEBUG MARKER: == conf-sanity test 80: mgc import reconnect race ======== 02:06:09 (1764745569) [ 993.357281] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 993.538278] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 994.781231] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 997.467248] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 998.817467] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 1000.543259] Lustre: DEBUG MARKER: oleg118-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1001.727778] Lustre: DEBUG MARKER: oleg118-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 1003.976117] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1004.002618] LustreError: 79340:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1004.008722] LustreError: 79340:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 1006.103332] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 1008.078346] Lustre: DEBUG MARKER: oleg118-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1009.435225] Lustre: 80080:0:(genops.c:1791:obd_export_evict_by_uuid()) MGS: evicting 8b157ccd-2156-4314-b636-3ba76f1882d5 at adminstrative request [ 1012.705865] LustreError: MGC192.168.201.118@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1012.712110] Lustre: *** cfs_fail_loc=906, val=2147483648*** [ 1021.921168] Lustre: MGS: Client 8b157ccd-2156-4314-b636-3ba76f1882d5 (at 0@lo) reconnecting [ 1021.926141] Lustre: Evicted from MGS (at 0@lo) after server handle changed from 0x889caba21dd462f1 to 0x889caba21dd46583 [ 1021.932201] Lustre: MGC192.168.201.118@tcp: Connection restored to 0@lo (at 0@lo) [ 1041.994371] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1042.024638] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1042.075564] Lustre: lustre-OST0001: new disk, initializing [ 1042.077545] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 1042.102366] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 1042.105235] Lustre: Skipped 2 previous similar messages [ 1044.211539] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 1047.524579] Lustre: DEBUG MARKER: oleg118-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid [ 1048.072096] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 1048.076204] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 1048.113104] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 1053.153226] LustreError: lustre-OST0001-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1053.153955] Lustre: lustre-OST0001-osc-MDT0001: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1053.156359] LustreError: Skipped 1 previous similar message [ 1053.160264] Lustre: Skipped 1 previous similar message [ 1053.162639] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 1053.166352] Lustre: Skipped 3 previous similar messages [ 1054.638073] LustreError: 81216:0:(obd_class.h:479:obd_check_dev()) Device 25 not setup [ 1054.640133] LustreError: 81216:0:(obd_class.h:479:obd_check_dev()) Skipped 13 previous similar messages [ 1054.662467] Lustre: server umount lustre-OST0001 complete [ 1058.274368] LustreError: 81140:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1058.279070] LustreError: 81140:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 1061.961700] Lustre: server umount lustre-OST0000 complete [ 1063.393072] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1063.396694] Lustre: Skipped 3 previous similar messages [ 1069.278421] Lustre: server umount lustre-MDT0000 complete [ 1070.545269] LustreError: 78460:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1764745649 with bad export cookie 9843931598298572163 [ 1070.548141] LustreError: MGC192.168.201.118@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1070.548396] LustreError: 78460:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 1070.670693] Lustre: server umount lustre-MDT0001 complete [ 1073.548073] Lustre: DEBUG MARKER: == conf-sanity test 81: sparse OST indexing ============== 02:07:32 (1764745652) [ 1074.103543] Lustre: DEBUG MARKER: SKIP: conf-sanity test_81 needs >= 3 OSTs [ 1074.714961] Lustre: DEBUG MARKER: == conf-sanity test 82a: specify OSTs for file (succeed) or directory (succeed) ========================================================== 02:07:33 (1764745653) [ 1075.308548] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82a needs >= 3 OSTs [ 1075.903987] Lustre: DEBUG MARKER: == conf-sanity test 82b: specify OSTs for file with --pool and --ost-list options ========================================================== 02:07:34 (1764745654) [ 1076.433239] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82b needs >= 4 OSTs [ 1077.084113] Lustre: DEBUG MARKER: == conf-sanity test 82c: specify sparse OSTs for setstripe (should not crash) ========================================================== 02:07:35 (1764745655) [ 1077.654742] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82c needs >= 4 OSTs [ 1078.255928] Lustre: DEBUG MARKER: == conf-sanity test 83: ENOSPACE on OST doesn't cause message VFS: Busy inodes after unmount ... ========================================================== 02:07:36 (1764745656) [ 1078.780917] Lustre: DEBUG MARKER: mount the OST /dev/mapper/ost1_flakey as a ldiskfs filesystem [ 1080.576615] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1080.923396] loop: module loaded [ 1080.928450] loop0: detected capacity change from 0 to 4294967296 [ 1080.940925] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 1081.484405] Lustre: DEBUG MARKER: run llverfs in partial mode on the OST ldiskfs /mnt/lustre-ost1 [ 1082.588033] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing run_llverfs /mnt/lustre-ost1 -vpl no [ 1083.958348] Lustre: DEBUG MARKER: unmount the OST /dev/mapper/ost1_flakey [ 1086.778586] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1088.086331] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1089.421622] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1090.793116] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1093.136344] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1093.160085] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1093.172086] LustreError: 84483:0:(obd_config.c:783:class_setup()) setup lustre-OST0000-osd failed (-28) [ 1093.174451] LustreError: 84483:0:(obd_mount.c:193:lustre_start_simple()) lustre-OST0000-osd setup error -28 [ 1093.176689] LustreError: 84483:0:(tgt_mount.c:2447:server_fill_super()) Unable to start osd on /dev/mapper/ost1_flakey: -28 [ 1093.179300] LustreError: 84483:0:(super25.c:199:lustre_fill_super()) llite: Unable to mount : rc = -28 [ 1099.693146] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_hostid [ 1103.106152] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing load_modules_local [ 1106.736412] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1109.029399] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1111.137268] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1113.429981] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1115.923044] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1115.948573] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1116.038125] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 1116.050946] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1116.095544] Lustre: lustre-MDT0000: new disk, initializing [ 1116.118156] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1116.125396] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1117.551375] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 1121.909081] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1121.932581] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1121.965676] Lustre: 87320:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 1122.069616] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 1123.472760] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 1126.714812] Lustre: DEBUG MARKER: oleg118-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1127.974188] Lustre: DEBUG MARKER: oleg118-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 1130.356897] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1130.381897] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1131.355592] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 1132.650855] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 1135.856404] Lustre: DEBUG MARKER: oleg118-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1137.334509] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 1137.422698] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 1138.837035] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 1138.923117] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 1141.729244] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 1141.730236] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1141.731542] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1141.736506] Lustre: Skipped 1 previous similar message [ 1141.738968] Lustre: Skipped 6 previous similar messages [ 1145.644102] LustreError: 90257:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 1145.646476] LustreError: 90257:0:(obd_class.h:479:obd_check_dev()) Skipped 25 previous similar messages [ 1145.689204] Lustre: server umount lustre-OST0000 complete [ 1153.113698] Lustre: server umount lustre-MDT0000 complete [ 1154.380447] LustreError: 87311:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1764745733 with bad export cookie 9843931598298572758 [ 1154.384684] LustreError: MGC192.168.201.118@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1154.501957] Lustre: server umount lustre-MDT0001 complete [ 1157.332684] Lustre: DEBUG MARKER: == conf-sanity test 84: check recovery_hard_time ========= 02:08:55 (1764745735) [ 1159.706395] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1159.859106] LustreError: 91330:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1161.214777] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 1164.238370] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1165.767764] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 1167.592168] Lustre: DEBUG MARKER: oleg118-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1168.755952] Lustre: DEBUG MARKER: oleg118-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 1171.175926] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1173.369445] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 1175.176624] Lustre: DEBUG MARKER: oleg118-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1176.613270] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 1185.884240] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 9 sec [ 1187.180974] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 1187.244968] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 1189.510967] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1189.532772] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1189.590582] Lustre: lustre-OST0001: new disk, initializing [ 1189.593204] Lustre: Skipped 2 previous similar messages [ 1189.595682] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 1189.597570] Lustre: Skipped 3 previous similar messages [ 1191.476678] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 1194.617634] Lustre: DEBUG MARKER: oleg118-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid [ 1195.993011] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid 50 [ 1196.544233] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 1196.546656] Lustre: Skipped 2 previous similar messages [ 1196.548944] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 1196.553920] Lustre: Skipped 1 previous similar message [ 1196.578884] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 1197.081169] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid in FULL state after 1 sec [ 1198.401486] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [ 1198.480505] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 1208.224989] Lustre: DEBUG MARKER: mds1 REPLAY BARRIER on lustre-MDT0000 [ 1213.744371] Lustre: Failing over lustre-MDT0000 [ 1214.118556] LustreError: 96284:0:(obd_class.h:479:obd_check_dev()) Device 28 not setup [ 1214.120586] LustreError: 96284:0:(obd_class.h:479:obd_check_dev()) Skipped 17 previous similar messages [ 1214.154840] Lustre: server umount lustre-MDT0000 complete [ 1215.117593] LustreError: 91324:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 192.168.201.18@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1215.122833] LustreError: 91324:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 1215.456246] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1215.459012] LustreError: Skipped 2 previous similar messages [ 1215.460234] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1215.463447] Lustre: Skipped 2 previous similar messages [ 1220.226668] LustreError: 93400:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 192.168.201.18@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1220.232415] LustreError: 93400:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 6 previous similar messages [ 1229.013086] LDISKFS-fs (dm-0): 4 truncates cleaned up [ 1229.015416] LDISKFS-fs (dm-0): recovery complete [ 1229.019105] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1229.060440] LustreError: MGC192.168.201.118@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1229.181196] Lustre: lustre-MDT0000: in recovery but waiting for the first client to connect [ 1230.467968] Lustre: lustre-MDT0000: Will be in recovery for at least 1:00, or until 3 clients reconnect [ 1230.470020] Lustre: 91325:0:(ldlm_lib.c:2068:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 1230.501779] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 1234.401799] Lustre: 91330:0:(ldlm_lib.c:2068:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 1234.405067] Lustre: 91330:0:(ldlm_lib.c:2068:extend_recovery_timer()) Skipped 1 previous similar message [ 1234.410427] Lustre: lustre-MDT0000-lwp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 1234.426861] LustreError: 97032:0:(ldlm_lib.c:2681:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 1234.743144] LustreError: 97032:0:(ldlm_lib.c:2681:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 1235.056960] LustreError: 97032:0:(ldlm_lib.c:2681:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 1235.060860] LustreError: 97032:0:(ldlm_lib.c:2681:replay_request_or_update()) Skipped 1 previous similar message [ 1235.375086] LustreError: 97032:0:(ldlm_lib.c:2681:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 1235.377779] LustreError: 97032:0:(ldlm_lib.c:2681:replay_request_or_update()) Skipped 1 previous similar message [ 1235.697266] Lustre: 97032:0:(ldlm_lib.c:2068:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 1235.702227] Lustre: 97032:0:(ldlm_lib.c:2068:extend_recovery_timer()) Skipped 8 previous similar messages [ 1236.329768] LustreError: 97032:0:(ldlm_lib.c:2681:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 1236.333103] LustreError: 97032:0:(ldlm_lib.c:2681:replay_request_or_update()) Skipped 3 previous similar messages [ 1236.647065] LustreError: 97032:0:(ldlm_lib.c:2681:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 1236.650624] LustreError: 97032:0:(ldlm_lib.c:2681:replay_request_or_update()) Skipped 3 previous similar messages [ 1237.903873] Lustre: 97032:0:(ldlm_lib.c:2068:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 1237.907602] Lustre: 97032:0:(ldlm_lib.c:2068:extend_recovery_timer()) Skipped 6 previous similar messages [ 1238.536032] LustreError: 97032:0:(ldlm_lib.c:2681:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 1238.540470] LustreError: 97032:0:(ldlm_lib.c:2681:replay_request_or_update()) Skipped 6 previous similar messages [ 1238.855089] LustreError: 97032:0:(ldlm_lib.c:2681:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 1238.857779] LustreError: 97032:0:(ldlm_lib.c:2681:replay_request_or_update()) Skipped 6 previous similar messages [ 1241.985451] Lustre: 97032:0:(ldlm_lib.c:2068:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 1241.988972] Lustre: 97032:0:(ldlm_lib.c:2068:extend_recovery_timer()) Skipped 12 previous similar messages [ 1242.617556] LustreError: 97032:0:(ldlm_lib.c:2681:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 1242.620917] LustreError: 97032:0:(ldlm_lib.c:2681:replay_request_or_update()) Skipped 12 previous similar messages [ 1242.935084] LustreError: 97032:0:(ldlm_lib.c:2681:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 1242.938086] LustreError: 97032:0:(ldlm_lib.c:2681:replay_request_or_update()) Skipped 12 previous similar messages [ 1250.121223] Lustre: 97032:0:(ldlm_lib.c:2068:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 1250.125036] Lustre: 97032:0:(ldlm_lib.c:2068:extend_recovery_timer()) Skipped 25 previous similar messages [ 1250.753877] LustreError: 97032:0:(ldlm_lib.c:2681:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 1250.758590] LustreError: 97032:0:(ldlm_lib.c:2681:replay_request_or_update()) Skipped 25 previous similar messages [ 1251.071127] LustreError: 97032:0:(ldlm_lib.c:2681:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 1251.073646] LustreError: 97032:0:(ldlm_lib.c:2681:replay_request_or_update()) Skipped 25 previous similar messages [ 1266.369309] Lustre: 97032:0:(ldlm_lib.c:2068:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 1266.375027] Lustre: 97032:0:(ldlm_lib.c:2068:extend_recovery_timer()) Skipped 51 previous similar messages [ 1267.009541] LustreError: 97032:0:(ldlm_lib.c:2681:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 1267.012671] LustreError: 97032:0:(ldlm_lib.c:2681:replay_request_or_update()) Skipped 51 previous similar messages [ 1267.327154] LustreError: 97032:0:(ldlm_lib.c:2681:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 1267.329834] LustreError: 97032:0:(ldlm_lib.c:2681:replay_request_or_update()) Skipped 51 previous similar messages [ 1290.111127] Lustre: 97032:0:(ldlm_lib.c:2377:target_recovery_overseer()) lustre-MDT0000 recovery is aborted by hard timeout [ 1290.115629] Lustre: 97032:0:(ldlm_lib.c:2387:target_recovery_overseer()) recovery is aborted, evict exports in recovery [ 1290.117962] Lustre: 97032:0:(genops.c:1618:class_disconnect_stale_exports()) lustre-MDT0000: disconnect stale client 6bcaa5d3-4066-4694-9302-d0c4aadb99a7@192.168.201.18@tcp [ 1290.122086] Lustre: lustre-MDT0000: disconnecting 1 stale clients [ 1290.124043] Lustre: 97032:0:(ldlm_lib.c:1897:abort_req_replay_queue()) @@@ aborted: req@00000000dc627930 x1850469707220352/t0(8589934775) o101->6bcaa5d3-4066-4694-9302-d0c4aadb99a7@192.168.201.18@tcp:575/0 lens 592/0 e 0 to 0 dl 1764745880 ref 1 fl Complete:/604/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:0 [ 1290.133795] Lustre: lustre-MDT0000-osd: cancel update llog [0x200000400:0x1:0x0] [ 1290.139613] Lustre: lustre-MDT0000: Denying connection for new client 6bcaa5d3-4066-4694-9302-d0c4aadb99a7 (at 192.168.201.18@tcp), waiting for 3 known clients (2 recovered, 0 in progress, and 1 evicted) to recover in 0:00 [ 1290.141268] Lustre: lustre-MDT0001-osp-MDT0000: cancel update llog [0x240000401:0x1:0x0] [ 1290.159997] Lustre: lustre-MDT0000-osp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 1290.161813] Lustre: lustre-MDT0000: Recovery over after 1:00, of 3 clients 2 recovered and 1 was evicted. [ 1290.163014] Lustre: Skipped 2 previous similar messages [ 1290.180244] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:46 to 0x2c0000401:545) [ 1290.180284] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:79 to 0x280000401:577) [ 1297.376297] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 1297.380754] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1297.388248] Lustre: Skipped 3 previous similar messages [ 1297.391633] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1297.393910] Lustre: Skipped 4 previous similar messages [ 1303.148092] LustreError: 97691:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 1303.150881] LustreError: 97691:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 1303.169363] Lustre: server umount lustre-OST0000 complete [ 1304.034275] LustreError: 92926:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1304.041040] LustreError: 92926:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 11 previous similar messages [ 1310.608322] Lustre: server umount lustre-OST0001 complete [ 1318.113541] Lustre: server umount lustre-MDT0000 complete [ 1319.389093] LustreError: 91310:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1764745898 with bad export cookie 9843931598298623942 [ 1319.392327] LustreError: MGC192.168.201.118@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1319.396476] LustreError: 91310:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 8 previous similar messages [ 1319.505915] Lustre: server umount lustre-MDT0001 complete [ 1322.240253] Lustre: DEBUG MARKER: == conf-sanity test 85: osd_ost init: fail ea_fid_set ==== 02:11:40 (1764745900) [ 1324.769416] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1324.772433] Lustre: *** cfs_fail_loc=197, val=0*** [ 1340.255464] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 1345.375393] LustreError: 98988:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.201.118@tcp: failed processing log, type 4: rc = -110 [ 1350.502914] Lustre: *** cfs_fail_loc=197, val=0*** [ 1350.504544] Lustre: Skipped 65 previous similar messages [ 1370.975234] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1370.977694] Lustre: Skipped 7 previous similar messages [ 1372.999262] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 1374.851344] Lustre: DEBUG MARKER: oleg118-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1375.582093] Lustre: server umount lustre-OST0000 complete [ 1378.336473] Lustre: DEBUG MARKER: == conf-sanity test 86: Replacing mkfs.lustre -G option == 02:12:37 (1764745957) [ 1391.791850] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1391.929289] LustreError: 100495:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1391.933956] LustreError: 100495:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 1393.136361] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 1396.151768] Lustre: DEBUG MARKER: oleg118-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1398.152411] LustreError: 100526:0:(osp_object.c:617:osp_attr_get()) lustre-MDT0001-osp-MDT0000: osp_attr_get update error [0x200000009:0x1:0x0]: rc = -5 [ 1398.155435] LustreError: 100526:0:(lod_sub_object.c:917:lod_sub_prep_llog()) lustre-MDT0000-mdtlov: can't get id from catalogs: rc = -5 [ 1398.158641] LustreError: 100526:0:(lod_dev.c:510:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 6, retries 0, failed: rc = -5 [ 1398.285326] Lustre: server umount lustre-MDT0000 complete [ 1403.640848] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1409.074652] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_hostid [ 1412.216130] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing load_modules_local [ 1415.347288] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1417.426046] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1419.555397] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1421.757470] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1424.117666] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1424.140984] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1424.226683] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 1424.240388] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1424.278762] Lustre: lustre-MDT0000: new disk, initializing [ 1424.311161] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1425.714272] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 1430.079337] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1430.102374] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1430.138381] Lustre: 104569:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 1430.262582] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 1431.700685] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 1435.030976] Lustre: DEBUG MARKER: oleg118-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1436.362098] Lustre: DEBUG MARKER: oleg118-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 1438.727363] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1438.757496] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1440.891372] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 1442.974020] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 1443.976605] Lustre: DEBUG MARKER: oleg118-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1445.367951] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 1445.446474] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 1446.787305] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 1446.870159] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 1448.416751] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 1448.418644] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1448.420537] LustreError: Skipped 3 previous similar messages [ 1448.423753] Lustre: Skipped 5 previous similar messages [ 1448.424353] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1448.428836] Lustre: Skipped 8 previous similar messages [ 1449.518116] LustreError: 107505:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 1449.521075] LustreError: 107505:0:(obd_class.h:479:obd_check_dev()) Skipped 41 previous similar messages [ 1449.565597] Lustre: server umount lustre-OST0000 complete [ 1465.311148] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 1465.386357] Lustre: server umount lustre-MDT0000 complete [ 1466.669358] LustreError: 104560:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1764746045 with bad export cookie 9843931598298634547 [ 1466.672956] LustreError: 104560:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1466.673602] LustreError: MGC192.168.201.118@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1466.793917] Lustre: server umount lustre-MDT0001 complete [ 1469.719836] Lustre: DEBUG MARKER: == conf-sanity test 87: check if MDT inode can hold EAs with N stripes properly ========================================================== 02:14:08 (1764746048) [ 1472.541978] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing unload_modules_local [ 1473.618194] Key type lgssc unregistered [ 1473.740391] LNet: 108633:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1473.742599] LNetError: 108633:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1473.749407] LNet: Removed LNI 192.168.201.118@tcp [ 1474.043900] Key type .llcrypt unregistered [ 1474.045041] Key type ._llcrypt unregistered [ 1479.689375] Key type ._llcrypt registered [ 1479.690405] Key type .llcrypt registered [ 1479.725025] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_hostid [ 1486.188608] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing load_modules_local [ 1486.589991] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1486.615674] alg: No test for adler32 (adler32-zlib) [ 1487.490770] Lustre: Lustre: Build Version: 2.17.0_RC1_2_g29d2772 [ 1487.582187] LNet: Added LNI 192.168.201.118@tcp [8/256/0/180] [ 1489.167120] Key type lgssc registered [ 1489.523733] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1492.528111] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1494.945695] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1497.056543] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1499.305157] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1501.079202] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1503.268693] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1503.287560] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1503.293062] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1504.380228] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 1504.394105] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1504.439650] Lustre: lustre-MDT0000: new disk, initializing [ 1504.466885] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1504.473656] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1505.897595] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 1510.077866] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1510.103288] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1510.220245] Lustre: lustre-OST0000: new disk, initializing [ 1510.222155] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1510.223726] Lustre: Skipped 1 previous similar message [ 1510.244846] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1512.158170] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 1516.331234] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1516.357915] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1516.405906] Lustre: lustre-OST0001: new disk, initializing [ 1516.408317] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 1516.434287] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 1516.677051] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 1516.680333] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 1516.683771] Lustre: Skipped 1 previous similar message [ 1516.690806] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x280000400 [ 1516.693287] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 1518.407266] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 1527.264954] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1527.265935] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1527.269892] Lustre: Skipped 1 previous similar message [ 1528.510191] LustreError: 114182:0:(obd_class.h:479:obd_check_dev()) Device 17 not setup [ 1528.559380] Lustre: server umount lustre-MDT0000 complete [ 1530.616262] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: (null) [ 1532.199411] LustreError: 111939:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1764746111 with bad export cookie 5644033986144630684 [ 1532.201676] LustreError: MGC192.168.201.118@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1532.203730] LustreError: 111939:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 1532.221087] LustreError: 114777:0:(obd_class.h:479:obd_check_dev()) Device 12 not setup [ 1532.223029] LustreError: 114777:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 1532.238886] Lustre: server umount lustre-OST0000 complete [ 1533.507791] LustreError: 114977:0:(obd_class.h:479:obd_check_dev()) Device 2 not setup [ 1533.511145] LustreError: 114977:0:(obd_class.h:479:obd_check_dev()) Skipped 1 previous similar message [ 1533.548355] Lustre: server umount lustre-OST0001 complete [ 1536.514163] Lustre: DEBUG MARKER: == conf-sanity test 88: check the default mount options can be overridden ========================================================== 02:15:15 (1764746115) [ 1539.266517] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing load_modules_local [ 1542.180709] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1542.546899] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1544.358842] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1544.709868] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1546.984092] Lustre: DEBUG MARKER: == conf-sanity test 89: check tunefs --param and --erase-param{s} options ========================================================== 02:15:25 (1764746125) [ 1551.148700] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1551.807337] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1552.525462] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1553.221929] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1553.552275] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1554.209996] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1559.990726] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_hostid [ 1563.165240] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing load_modules_local [ 1566.565367] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1568.815196] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1570.998672] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1573.219277] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1575.605874] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1575.636522] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1575.725256] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 1575.737471] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1575.775734] Lustre: lustre-MDT0000: new disk, initializing [ 1575.800076] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1575.807311] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1575.810480] Lustre: Skipped 1 previous similar message [ 1577.191868] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 1581.396640] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1581.418908] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1581.454981] Lustre: 120291:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 1581.480147] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 1581.482177] Lustre: Skipped 1 previous similar message [ 1581.520634] Lustre: lustre-MDT0001: new disk, initializing [ 1581.555093] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 1581.569504] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 1581.572778] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 1582.938546] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 1586.040199] Lustre: DEBUG MARKER: oleg118-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1587.184579] Lustre: DEBUG MARKER: oleg118-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 1589.450621] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1589.479391] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1589.598715] Lustre: lustre-OST0000: new disk, initializing [ 1589.601815] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1589.633928] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1591.598632] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 1594.357115] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 1594.362642] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 1594.392141] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 1594.735631] Lustre: DEBUG MARKER: oleg118-client.virtnet: executing wait_import_state_mount (FULL|IDLE) failover.node=192.0.2.254@tcp0,osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1596.085136] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 1596.151852] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 1597.415577] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 1597.499851] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 1599.458912] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1599.458949] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1599.459974] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1599.459979] Lustre: Skipped 1 previous similar message [ 1599.461736] LustreError: Skipped 1 previous similar message [ 1599.465445] Lustre: Skipped 1 previous similar message [ 1600.172973] LustreError: 123230:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 1600.175819] LustreError: 123230:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 1600.213840] Lustre: server umount lustre-OST0000 complete [ 1602.016176] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1602.018833] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1602.023511] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1602.025156] Lustre: Skipped 1 previous similar message [ 1604.576180] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1604.577057] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1607.619163] LustreError: 123430:0:(obd_class.h:479:obd_check_dev()) Device 22 not setup [ 1607.621957] LustreError: 123430:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 1607.655556] Lustre: server umount lustre-MDT0000 complete [ 1609.080356] LustreError: 120282:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1764746188 with bad export cookie 5644033986144631853 [ 1609.085783] LustreError: 120282:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1609.085908] LustreError: MGC192.168.201.118@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1609.206649] Lustre: server umount lustre-MDT0001 complete [ 1612.167931] Lustre: DEBUG MARKER: == conf-sanity test 90a: check max_mod_rpcs_in_flight is enforced ========================================================== 02:16:30 (1764746190) [ 1614.526077] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1614.705435] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1616.134773] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 1619.220664] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1620.857870] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 1622.737693] Lustre: DEBUG MARKER: oleg118-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1624.039980] Lustre: DEBUG MARKER: oleg118-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 1626.414057] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1628.479546] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 1630.383234] Lustre: DEBUG MARKER: oleg118-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1643.184083] Lustre: *** cfs_fail_loc=159, val=0*** [ 1659.542076] Lustre: lustre-MDT0000: Client 9f9c102f-f34a-4571-8cb6-2087b6542a33 (at 192.168.201.18@tcp) reconnecting [ 1660.125342] Lustre: *** cfs_fail_loc=159, val=0*** [ 1660.127440] Lustre: Skipped 6 previous similar messages [ 1676.435593] Lustre: lustre-MDT0000: Client 9f9c102f-f34a-4571-8cb6-2087b6542a33 (at 192.168.201.18@tcp) reconnecting [ 1679.844909] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1679.845490] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1679.848044] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1679.853123] Lustre: Skipped 1 previous similar message [ 1679.856332] Lustre: Skipped 1 previous similar message [ 1683.820129] LustreError: 126814:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 1683.822726] LustreError: 126814:0:(obd_class.h:479:obd_check_dev()) Skipped 13 previous similar messages [ 1683.858128] Lustre: server umount lustre-OST0000 complete [ 1685.983697] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1690.080334] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1690.080756] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1690.082643] Lustre: Skipped 1 previous similar message [ 1690.085712] Lustre: Skipped 1 previous similar message [ 1691.224638] Lustre: server umount lustre-MDT0000 complete [ 1692.508673] LustreError: 126715:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1764746271 with bad export cookie 5644033986144632721 [ 1692.514086] LustreError: MGC192.168.201.118@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1692.518289] LustreError: 126715:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1692.641279] Lustre: server umount lustre-MDT0001 complete [ 1696.251512] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing unload_modules_local [ 1697.272534] Key type lgssc unregistered [ 1697.415351] LNet: 127749:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1697.417828] LNetError: 127749:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1697.424332] LNet: Removed LNI 192.168.201.118@tcp [ 1697.729312] Key type .llcrypt unregistered [ 1697.730628] Key type ._llcrypt unregistered [ 1706.624907] Key type ._llcrypt registered [ 1706.625906] Key type .llcrypt registered [ 1706.664354] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing load_modules_local [ 1707.084206] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1707.090986] alg: No test for adler32 (adler32-zlib) [ 1707.941307] Lustre: Lustre: Build Version: 2.17.0_RC1_2_g29d2772 [ 1708.025166] LNet: Added LNI 192.168.201.118@tcp [8/256/0/180] [ 1709.599140] Key type lgssc registered [ 1709.923746] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1712.665595] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1712.673350] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1713.843932] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1715.170223] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 1718.126731] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1718.263137] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 1719.690323] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 1721.438720] Lustre: DEBUG MARKER: oleg118-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1722.649127] Lustre: DEBUG MARKER: oleg118-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 1724.954263] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1725.074346] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1727.019374] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 1728.871859] Lustre: DEBUG MARKER: oleg118-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1739.435976] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:42 to 0x280000401:65) [ 1749.185961] Lustre: *** cfs_fail_loc=159, val=0*** [ 1765.521440] Lustre: lustre-MDT0000: Client 94bc263b-0cb7-4f49-b6df-ba5170fd6891 (at 192.168.201.18@tcp) reconnecting [ 1766.234080] Lustre: *** cfs_fail_loc=159, val=0*** [ 1766.235534] Lustre: Skipped 1 previous similar message [ 1782.413870] Lustre: lustre-MDT0001: Client 94bc263b-0cb7-4f49-b6df-ba5170fd6891 (at 192.168.201.18@tcp) reconnecting [ 1782.958514] Lustre: *** cfs_fail_loc=159, val=0*** [ 1782.960407] Lustre: Skipped 2 previous similar messages [ 1799.311396] Lustre: lustre-MDT0001: Client 94bc263b-0cb7-4f49-b6df-ba5170fd6891 (at 192.168.201.18@tcp) reconnecting [ 1800.478993] Lustre: 131844:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 1801.319513] Lustre: *** cfs_fail_loc=159, val=0*** [ 1801.321551] Lustre: Skipped 4 previous similar messages [ 1817.748896] Lustre: lustre-MDT0000: Client 3c88827c-a6c5-4df3-b469-2a4242f3cfdc (at 192.168.201.18@tcp) reconnecting [ 1818.362759] Lustre: *** cfs_fail_loc=159, val=0*** [ 1818.365058] Lustre: Skipped 14 previous similar messages [ 1834.637914] Lustre: lustre-MDT0000: Client 3c88827c-a6c5-4df3-b469-2a4242f3cfdc (at 192.168.201.18@tcp) reconnecting [ 1835.172456] Lustre: 132138:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 1852.385516] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1852.388118] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1852.388976] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1852.389405] Lustre: Skipped 1 previous similar message [ 1857.508963] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1857.510940] Lustre: Skipped 2 previous similar messages [ 1858.158182] LustreError: 132262:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 1858.204076] Lustre: server umount lustre-OST0000 complete [ 1861.600103] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1861.603835] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1861.610513] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1862.625214] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1867.744080] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1867.745727] Lustre: Skipped 3 previous similar messages [ 1872.864283] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1872.867301] Lustre: Skipped 1 previous similar message [ 1874.399131] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 1874.436583] LustreError: 132463:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup [ 1874.438362] LustreError: 132463:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 1874.473294] Lustre: server umount lustre-MDT0000 complete [ 1875.808380] LustreError: 129101:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1764746455 with bad export cookie 1920730012052650 [ 1875.811751] LustreError: 129101:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 1 previous similar message [ 1875.813395] LustreError: MGC192.168.201.118@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1875.882024] LustreError: 132667:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [ 1875.883869] LustreError: 132667:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 1875.939940] Lustre: server umount lustre-MDT0001 complete [ 1879.396670] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing unload_modules_local [ 1880.494896] Key type lgssc unregistered [ 1880.646482] LNet: 133197:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1880.649388] LNetError: 133197:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1880.657577] LNet: Removed LNI 192.168.201.118@tcp [ 1880.992173] Key type .llcrypt unregistered [ 1880.993305] Key type ._llcrypt unregistered [ 1890.170916] Key type ._llcrypt registered [ 1890.172323] Key type .llcrypt registered [ 1890.212781] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing load_modules_local [ 1890.547416] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1890.586286] alg: No test for adler32 (adler32-zlib) [ 1891.439162] Lustre: Lustre: Build Version: 2.17.0_RC1_2_g29d2772 [ 1891.526582] LNet: Added LNI 192.168.201.118@tcp [8/256/0/180] [ 1893.103117] Key type lgssc registered [ 1893.467794] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1895.944896] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1895.950811] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1897.103556] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1898.339759] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 1901.143349] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1901.265614] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 1902.607503] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 1904.420949] Lustre: DEBUG MARKER: oleg118-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1905.621677] Lustre: DEBUG MARKER: oleg118-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 1907.903265] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1908.030719] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1909.928371] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 1911.144379] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:97) [ 1911.842774] Lustre: DEBUG MARKER: oleg118-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1922.405729] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:129) [ 1929.565232] Lustre: 136837:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 1930.222083] Lustre: 136885:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 1930.677482] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 in log params [ 1931.766361] Lustre: 137033:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify lustre/mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 (mode = 0) failed: rc = -17 [ 1932.068982] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=7 in log params [ 1932.071387] Lustre: Skipped 1 previous similar message [ 1932.383393] Lustre: 137131:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 1936.864580] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 1936.869129] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1936.877273] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1937.891682] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1937.892249] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1937.892576] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1939.446350] LustreError: 137229:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 1939.488030] Lustre: server umount lustre-OST0000 complete [ 1942.495887] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1942.498926] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1942.504184] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1942.506319] Lustre: Skipped 1 previous similar message [ 1948.128861] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1948.130641] Lustre: Skipped 3 previous similar messages [ 1953.249649] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1953.252862] Lustre: Skipped 1 previous similar message [ 1955.295158] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 1955.337061] LustreError: 137438:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup [ 1955.338874] LustreError: 137438:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 1955.361200] Lustre: server umount lustre-MDT0000 complete [ 1956.605355] LustreError: 134551:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1764746535 with bad export cookie 17373165064041784100 [ 1956.607709] LustreError: MGC192.168.201.118@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1956.608737] LustreError: 134551:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 1956.674283] LustreError: 137642:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [ 1956.677310] LustreError: 137642:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 1956.728499] Lustre: server umount lustre-MDT0001 complete [ 1960.274898] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing unload_modules_local [ 1961.326614] Key type lgssc unregistered [ 1961.463381] LNet: 138172:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1961.465557] LNetError: 138172:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1961.476366] LNet: Removed LNI 192.168.201.118@tcp [ 1961.789934] Key type .llcrypt unregistered [ 1961.791054] Key type ._llcrypt unregistered [ 1971.287108] Key type ._llcrypt registered [ 1971.288593] Key type .llcrypt registered [ 1971.331167] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing load_modules_local [ 1971.678637] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1971.733032] alg: No test for adler32 (adler32-zlib) [ 1972.658792] Lustre: Lustre: Build Version: 2.17.0_RC1_2_g29d2772 [ 1972.816395] LNet: Added LNI 192.168.201.118@tcp [8/256/0/180] [ 1974.455128] Key type lgssc registered [ 1975.176992] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1980.288728] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1980.302573] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1981.584446] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1983.925425] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 1988.453956] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1988.626379] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 1990.792519] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 1993.754449] Lustre: DEBUG MARKER: oleg118-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1995.895792] Lustre: DEBUG MARKER: oleg118-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 1999.684246] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1999.873433] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2002.927087] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 2006.069655] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:161) [ 2008.198256] Lustre: DEBUG MARKER: oleg118-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2013.230710] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:129) [ 2017.186982] Lustre: *** cfs_fail_loc=159, val=0*** [ 2017.241733] Lustre: Skipped 6 previous similar messages [ 2033.826215] Lustre: lustre-MDT0000: Client 705aad5a-7cb5-44ae-b5f8-4fd6961be4cb (at 192.168.201.18@tcp) reconnecting [ 2036.705604] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2036.708925] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2036.716074] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2038.754096] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2038.756108] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2038.758036] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2042.164510] LustreError: 141926:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 2042.226209] Lustre: server umount lustre-OST0000 complete [ 2044.896348] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2044.900490] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2044.906184] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2044.908427] Lustre: Skipped 1 previous similar message [ 2048.993629] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2048.996858] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2050.252406] LustreError: 142127:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup [ 2050.254854] LustreError: 142127:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 2050.307573] Lustre: server umount lustre-MDT0000 complete [ 2052.444046] LustreError: 139528:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1764746631 with bad export cookie 5856471427971606433 [ 2052.450671] LustreError: MGC192.168.201.118@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2052.452305] LustreError: 139528:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2052.549707] LustreError: 142328:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [ 2052.555515] LustreError: 142328:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 2052.680636] Lustre: server umount lustre-MDT0001 complete [ 2057.552429] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing unload_modules_local [ 2058.929464] Key type lgssc unregistered [ 2059.097523] LNet: 142859:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2059.102095] LNetError: 142859:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2059.118551] LNet: Removed LNI 192.168.201.118@tcp [ 2059.604973] Key type .llcrypt unregistered [ 2059.607098] Key type ._llcrypt unregistered [ 2071.564769] Key type ._llcrypt registered [ 2071.566496] Key type .llcrypt registered [ 2071.612949] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing load_modules_local [ 2071.996575] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2072.072507] alg: No test for adler32 (adler32-zlib) [ 2072.975925] Lustre: Lustre: Build Version: 2.17.0_RC1_2_g29d2772 [ 2073.098145] LNet: Added LNI 192.168.201.118@tcp [8/256/0/180] [ 2074.703152] Key type lgssc registered [ 2075.349165] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2079.531673] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 2079.544235] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2080.826401] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2082.872580] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 2087.546482] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2087.793942] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2090.070142] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 2093.131068] Lustre: DEBUG MARKER: oleg118-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2095.010065] Lustre: DEBUG MARKER: oleg118-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2098.679698] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2098.911875] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2101.676794] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 2104.431481] Lustre: DEBUG MARKER: oleg118-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2106.094233] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:170 to 0x280000401:193) [ 2112.233813] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:161) [ 2129.490581] Lustre: lustre-OST0000: evicting 008c82b5-a04d-485a-9ff4-cc19a5b807c2 (at 192.168.201.18@tcp) by administrative request [ 2129.493945] Lustre: Skipped 1 previous similar message [ 2157.536346] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2157.541589] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2157.548416] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2158.561018] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2158.562056] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2158.566903] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2160.369086] LustreError: 147033:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 2160.429542] Lustre: server umount lustre-OST0000 complete [ 2163.680668] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2163.682894] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2163.687912] Lustre: Skipped 1 previous similar message [ 2163.690616] Lustre: Skipped 1 previous similar message [ 2168.337144] LustreError: 147236:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup [ 2168.340086] LustreError: 147236:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 2168.389688] Lustre: server umount lustre-MDT0000 complete [ 2168.801032] LustreError: 144232:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2168.808880] LustreError: 144232:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 2170.179511] LustreError: 145859:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1764746749 with bad export cookie 17856887295091760558 [ 2170.180697] LustreError: MGC192.168.201.118@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2170.185592] LustreError: 145859:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2170.285720] LustreError: 147437:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [ 2170.289316] LustreError: 147437:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 2170.358062] Lustre: server umount lustre-MDT0001 complete [ 2175.254219] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing unload_modules_local [ 2176.516394] Key type lgssc unregistered [ 2176.702025] LNet: 147969:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2176.705455] LNetError: 147969:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2176.719457] LNet: Removed LNI 192.168.201.118@tcp [ 2177.139483] Key type .llcrypt unregistered [ 2177.141680] Key type ._llcrypt unregistered [ 2213.738132] Key type ._llcrypt registered [ 2213.739692] Key type .llcrypt registered [ 2213.803870] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing load_modules_local [ 2214.305646] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2214.317153] alg: No test for adler32 (adler32-zlib) [ 2215.192665] Lustre: Lustre: Build Version: 2.17.0_RC1_2_g29d2772 [ 2215.294397] LNet: Added LNI 192.168.201.118@tcp [8/256/0/180] [ 2216.887461] Key type lgssc registered [ 2217.466325] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2221.186601] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 2221.195669] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2222.445310] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2224.409511] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 2228.781435] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2229.025240] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2231.186216] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 2233.972885] Lustre: DEBUG MARKER: oleg118-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2235.789194] Lustre: DEBUG MARKER: oleg118-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2239.222699] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2239.270894] LustreError: 152013:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2239.278641] LustreError: 152013:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 2239.420573] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2242.253698] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 2244.591918] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:193) [ 2245.011985] Lustre: DEBUG MARKER: oleg118-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2247.657913] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:195 to 0x280000401:225) [ 2254.816559] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2254.820444] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2254.826500] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2257.893856] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2257.894259] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2257.903438] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2260.728226] LustreError: 152666:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 2260.783151] Lustre: server umount lustre-OST0000 complete [ 2263.010246] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2263.020670] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2263.023260] Lustre: Skipped 1 previous similar message [ 2268.129846] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2268.136496] Lustre: Skipped 1 previous similar message [ 2268.608725] LustreError: 152867:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup [ 2268.610900] LustreError: 152867:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 2268.641719] Lustre: server umount lustre-MDT0000 complete [ 2270.323366] LustreError: 152036:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1764746849 with bad export cookie 13322469725349794778 [ 2270.327362] LustreError: MGC192.168.201.118@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2270.328992] LustreError: 152036:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2270.399150] LustreError: 153070:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [ 2270.401593] LustreError: 153070:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 2270.465553] Lustre: server umount lustre-MDT0001 complete [ 2275.002778] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing unload_modules_local [ 2276.291232] Key type lgssc unregistered [ 2276.442458] LNet: 153600:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2276.449392] LNetError: 153600:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2276.459098] LNet: Removed LNI 192.168.201.118@tcp [ 2276.882704] Key type .llcrypt unregistered [ 2276.883768] Key type ._llcrypt unregistered [ 2288.098368] Key type ._llcrypt registered [ 2288.099705] Key type .llcrypt registered [ 2288.149866] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing load_modules_local [ 2288.623189] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2288.657798] alg: No test for adler32 (adler32-zlib) [ 2289.533388] Lustre: Lustre: Build Version: 2.17.0_RC1_2_g29d2772 [ 2289.637696] LNet: Added LNI 192.168.201.118@tcp [8/256/0/180] [ 2291.231223] Key type lgssc registered [ 2291.689474] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2294.782092] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2297.877363] Lustre: DEBUG MARKER: == conf-sanity test 100: check lshowmount lists MGS, MDT, OST and 0@lo ========================================================== 02:27:56 (1764746876) [ 2304.847618] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_hostid [ 2309.238569] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing load_modules_local [ 2313.697942] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2316.462729] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2319.210424] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2322.021927] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2325.106316] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2325.137784] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 2325.146545] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2326.274957] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2326.301690] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2326.349280] Lustre: lustre-MDT0000: new disk, initializing [ 2326.407899] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2326.424255] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2328.200846] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 2333.482418] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2333.527180] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2333.582117] Lustre: 157796:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2333.624944] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 2333.624963] Lustre: lustre-MDT0001: Not available for connect from 0@lo (not set up) [ 2333.627504] Lustre: Skipped 1 previous similar message [ 2333.672421] Lustre: lustre-MDT0001: new disk, initializing [ 2333.717444] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2333.741313] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 2333.745379] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2335.602624] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 2339.857680] Lustre: DEBUG MARKER: oleg118-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2341.832862] Lustre: DEBUG MARKER: oleg118-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2345.406166] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2345.461093] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2345.602456] Lustre: lustre-OST0000: new disk, initializing [ 2345.605854] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 2345.664204] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2348.332070] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 2350.106286] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 2350.111293] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2350.153900] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2352.522246] Lustre: DEBUG MARKER: oleg118-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2360.290342] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2360.292159] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2360.294766] LustreError: Skipped 1 previous similar message [ 2360.296299] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2360.310518] Lustre: Skipped 1 previous similar message [ 2364.529160] LustreError: 160380:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 2364.594844] Lustre: server umount lustre-OST0000 complete [ 2369.505114] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2369.508876] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2369.523192] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2369.525745] Lustre: Skipped 1 previous similar message [ 2370.528675] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2372.682545] LustreError: 160581:0:(obd_class.h:479:obd_check_dev()) Device 22 not setup [ 2372.687072] LustreError: 160581:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 2372.740550] Lustre: server umount lustre-MDT0000 complete [ 2374.848730] LustreError: 159541:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1764746954 with bad export cookie 6103999563659478842 [ 2374.849188] LustreError: MGC192.168.201.118@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2374.852969] LustreError: 159541:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 2374.936377] LustreError: 160783:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 2374.941740] LustreError: 160783:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 2375.062290] Lustre: server umount lustre-MDT0001 complete [ 2380.521482] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing unload_modules_local [ 2382.025086] Key type lgssc unregistered [ 2382.205899] LNet: 161315:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2382.214992] LNetError: 161315:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2382.223579] LNet: Removed LNI 192.168.201.118@tcp [ 2382.661760] Key type .llcrypt unregistered [ 2382.664304] Key type ._llcrypt unregistered [ 2395.268392] Key type ._llcrypt registered [ 2395.269792] Key type .llcrypt registered [ 2395.335892] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing load_modules_local [ 2395.749606] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2395.757788] alg: No test for adler32 (adler32-zlib) [ 2396.661584] Lustre: Lustre: Build Version: 2.17.0_RC1_2_g29d2772 [ 2396.771988] LNet: Added LNI 192.168.201.118@tcp [8/256/0/180] [ 2398.367479] Key type lgssc registered [ 2398.956475] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2402.854174] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 2402.866274] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2404.130779] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2406.106877] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 2410.523399] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2410.737412] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2412.647981] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 2415.445643] Lustre: DEBUG MARKER: oleg118-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2417.110782] Lustre: DEBUG MARKER: oleg118-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2420.035949] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2420.201386] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2422.793382] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 2425.259631] Lustre: DEBUG MARKER: oleg118-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2429.415206] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:3 to 0x280000401:33) [ 2438.114143] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 2438.122716] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2438.129600] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 2438.133383] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2438.140215] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 2438.557228] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2438.990557] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 2438.996196] Lustre: Skipped 1 previous similar message [ 2439.006503] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2439.012509] Lustre: Skipped 1 previous similar message [ 2439.015032] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 2439.018228] Lustre: Skipped 1 previous similar message [ 2439.021332] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2439.029347] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 2439.032421] Lustre: Skipped 1 previous similar message [ 2439.452868] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2439.861778] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2440.273269] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 2440.277974] Lustre: Skipped 2 previous similar messages [ 2440.285693] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2440.292688] Lustre: Skipped 2 previous similar messages [ 2440.295635] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 2440.300198] Lustre: Skipped 2 previous similar messages [ 2440.302568] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2440.308523] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 2440.311774] Lustre: Skipped 2 previous similar messages [ 2440.749064] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2441.219246] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2441.734486] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2442.177431] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2442.630329] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 2442.633647] Lustre: Skipped 4 previous similar messages [ 2442.643992] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2442.649738] Lustre: Skipped 4 previous similar messages [ 2442.653526] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 2442.656544] Lustre: Skipped 4 previous similar messages [ 2442.658799] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2442.665526] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 2442.668848] Lustre: Skipped 4 previous similar messages [ 2443.110142] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2443.575713] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2443.987463] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2444.390455] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2444.820238] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2445.246727] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2445.699724] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2446.168594] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2446.560231] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2446.978649] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 2446.982464] Lustre: Skipped 9 previous similar messages [ 2446.990793] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2446.996353] Lustre: Skipped 9 previous similar messages [ 2446.998329] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 2447.001482] Lustre: Skipped 9 previous similar messages [ 2447.003759] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2447.008833] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 2447.011901] Lustre: Skipped 9 previous similar messages [ 2447.392974] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2447.751215] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2448.138898] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2448.492331] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2448.862922] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2449.256987] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2449.615812] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2449.969218] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2450.319524] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2450.693946] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2451.066074] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2451.487972] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2451.894088] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2452.289617] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2452.711996] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2453.137243] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2453.536609] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2453.937781] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2454.320280] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2454.716356] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2455.091674] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 2455.095395] Lustre: Skipped 20 previous similar messages [ 2455.102632] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2455.108548] Lustre: Skipped 20 previous similar messages [ 2455.110927] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 2455.114039] Lustre: Skipped 20 previous similar messages [ 2455.116732] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2455.123542] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 2455.126377] Lustre: Skipped 20 previous similar messages [ 2455.496392] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2455.876452] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2456.261617] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2456.659870] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2457.043059] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2457.442043] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2457.821884] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2458.219477] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2458.633046] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2459.033462] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2459.427857] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2459.872705] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2460.270035] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2460.689154] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2461.135716] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2461.558570] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2461.971888] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2462.346916] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2462.783749] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2463.206687] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2463.600864] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2464.073313] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2464.536036] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2464.930500] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2465.318524] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2465.696176] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2466.050055] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2466.435709] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2466.807194] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2467.149879] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2467.500094] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2467.851862] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2468.208338] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2468.553590] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2468.907195] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2469.258746] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2469.635036] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2470.004347] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2470.370043] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2470.756180] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2471.139552] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 2471.143276] Lustre: Skipped 40 previous similar messages [ 2471.151218] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2471.156747] Lustre: Skipped 40 previous similar messages [ 2471.159160] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 2471.162208] Lustre: Skipped 40 previous similar messages [ 2471.164570] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2471.169353] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 2471.171779] Lustre: Skipped 40 previous similar messages [ 2471.551966] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2471.915811] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2472.271580] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2472.671195] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2473.049922] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2473.419923] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2473.828453] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2474.219177] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2474.622928] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2474.987808] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2475.366749] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2475.739435] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2476.098103] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2476.456558] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2476.819735] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2477.183538] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2477.600611] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2478.020882] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2478.478585] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2478.893285] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2479.329251] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2479.731904] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2480.132666] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2480.553933] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2480.946494] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2481.367907] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2481.761778] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2482.208330] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2482.619171] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2483.008636] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2483.388107] LustreError: 162708:0:(osp_precreate.c:651:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 2483.393763] LustreError: 162708:0:(osp_precreate.c:1355:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 2483.395894] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2483.779682] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2484.146578] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2484.511689] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2484.897333] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2485.289747] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2485.720793] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2486.125108] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2486.519776] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2486.945994] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2487.317566] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2487.674642] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2488.051936] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2488.401122] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2488.767900] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2489.220166] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2489.612868] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2490.043223] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2490.453630] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2490.840838] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2491.264932] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2491.688994] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2492.096648] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2492.471409] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2492.844079] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2493.218716] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2493.581436] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2493.949005] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2494.305182] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2494.657777] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2495.002755] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2495.361661] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2495.717667] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2496.065486] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2496.434181] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2496.800206] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2497.227408] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2497.640797] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2498.036856] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2498.455346] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2498.863040] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2499.233491] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2499.655636] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2500.046705] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2500.425862] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2500.818648] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2501.211649] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2501.590346] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2502.013423] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2502.397800] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2502.771399] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2503.189177] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 2503.192630] Lustre: Skipped 81 previous similar messages [ 2503.201308] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2503.207661] Lustre: Skipped 81 previous similar messages [ 2503.210593] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 2503.214625] Lustre: Skipped 81 previous similar messages [ 2503.216708] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2503.222755] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 2503.227215] Lustre: Skipped 81 previous similar messages [ 2503.627169] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2504.057335] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2504.482908] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2504.898743] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2505.313494] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2505.738996] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2506.140543] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2506.525344] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2506.917065] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2507.332690] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2507.715680] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2508.090717] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2508.494898] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2508.893430] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2509.280673] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2509.716437] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2510.108190] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2510.539216] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2510.949148] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2511.350944] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2511.760212] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2512.167718] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2512.529384] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2512.930194] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2513.344161] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2513.736051] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2514.149171] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2514.565567] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2514.978489] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2515.412500] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2515.809345] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2516.181440] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2516.584125] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2516.993623] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2517.382207] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2517.850073] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2518.272800] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2518.669071] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2519.107370] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2519.523030] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2519.934879] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2520.329833] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2520.732399] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2521.158598] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2521.545518] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2521.941200] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2522.319612] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2522.691735] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2523.059831] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2523.407890] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2523.769071] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2524.155234] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2524.532410] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2524.963214] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2525.369231] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2525.764850] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2526.144617] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2526.525470] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2526.911851] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2527.300256] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2527.688749] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2528.057761] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2528.418907] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2528.797619] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2529.180965] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2529.614344] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2530.032725] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2530.455269] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2530.891837] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2531.343180] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2531.757755] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2532.176742] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2532.646377] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2533.104354] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2533.550779] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2533.995820] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2534.411083] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2534.839184] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2535.234440] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2535.627277] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2536.044384] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2536.458991] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2536.867262] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2537.256080] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2537.654628] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2538.052813] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2538.453677] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2538.889851] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2539.291740] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2539.707579] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2540.151612] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2540.547734] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2540.937547] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2541.342154] LustreError: 162708:0:(osp_precreate.c:651:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 2541.347065] LustreError: 162708:0:(osp_precreate.c:1355:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 2541.348225] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2541.747415] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2542.145352] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2542.575264] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2542.986352] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2543.353880] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2543.715572] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2544.067074] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2544.436100] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2544.827587] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2545.193406] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2545.557689] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2545.922627] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2546.291266] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2546.682920] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2547.067751] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2547.462683] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2547.895599] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2548.275989] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2548.680239] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2549.059308] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2549.423553] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2549.791553] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2550.185070] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2550.552745] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2550.904772] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2551.268607] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2551.635069] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2551.975919] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2552.321571] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2552.679821] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2553.026247] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2553.386329] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2553.743508] LustreError: 162708:0:(osp_precreate.c:651:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 2553.748270] LustreError: 162708:0:(osp_precreate.c:1355:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 2553.749221] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2554.115414] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2554.464935] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2554.823167] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2555.204597] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2555.628432] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2556.016824] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2556.392258] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2556.782547] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2557.187806] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2557.539688] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2557.934546] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2558.367296] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2558.760702] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2559.186493] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2559.608692] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2560.048442] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2560.442710] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2560.919624] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2561.356710] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2561.774672] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2562.192839] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2562.593257] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2563.013360] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2563.427743] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2563.858258] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2564.261256] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2564.663620] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2565.036856] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2565.456932] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2565.884676] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2566.271862] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2566.634296] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2567.021794] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2567.406331] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 2567.409176] Lustre: Skipped 160 previous similar messages [ 2567.419962] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2567.426266] Lustre: Skipped 160 previous similar messages [ 2567.429563] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 2567.433578] Lustre: Skipped 160 previous similar messages [ 2567.435801] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2567.440829] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 2567.443471] Lustre: Skipped 160 previous similar messages [ 2567.817839] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2568.187324] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2568.591865] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2568.985336] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2569.380176] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2569.740826] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2570.097749] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2570.452166] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2570.816644] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2571.138704] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2571.469054] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2571.790264] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2572.145672] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2572.520371] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2572.896394] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2573.241759] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2573.587477] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2573.928707] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2574.254930] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2574.581432] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2574.910089] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2575.222340] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2575.528316] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2575.850855] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2576.205167] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2576.522258] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2576.849818] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2577.180567] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2577.526759] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2577.859654] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2578.196683] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2578.527831] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2578.885351] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2579.217318] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2579.555398] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2579.881181] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2580.239657] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2580.571064] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2580.927900] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2581.268867] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2581.636121] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2581.990681] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2582.341115] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2582.699362] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2583.073857] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2583.453168] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2583.825441] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2584.218753] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2584.592533] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2584.974336] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2585.333711] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2585.708866] LustreError: 162708:0:(osp_precreate.c:651:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 2585.714201] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2585.714217] LustreError: 162708:0:(osp_precreate.c:1355:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 2586.068954] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2586.428198] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2586.795561] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2587.132896] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2587.477527] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2587.846833] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2588.194648] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2588.577312] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2588.954380] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2589.344153] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2589.707324] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2590.088621] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2590.463178] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2590.819144] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2591.195402] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2591.600158] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2591.982244] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2592.395465] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2592.766686] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2593.146337] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2593.524246] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2593.910878] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2594.315089] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2594.687559] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2595.042075] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2595.410660] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2595.792919] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2596.161224] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2596.527098] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2596.879743] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2597.226223] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2597.573544] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2597.961210] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2598.353257] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2598.735459] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2599.106896] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2599.466985] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2599.889575] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2600.305583] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2600.696433] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2601.092131] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2601.484386] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2601.886251] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2602.289784] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2602.708286] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2603.125728] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2603.556352] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2603.962891] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2604.445871] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2604.875497] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2605.278851] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2605.691354] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2606.116720] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2606.533195] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2606.969496] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2607.389836] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2607.825408] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2608.247685] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2608.655495] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2609.115307] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2609.535314] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2609.983712] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2610.459651] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2610.948505] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2611.396428] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2611.826814] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2612.230465] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2612.649088] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2613.055440] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2613.474178] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2613.875383] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2614.291105] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2614.692042] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2615.059398] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2615.442228] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2615.842843] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2616.206819] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2616.557665] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2616.888780] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2617.230689] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2617.584154] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2617.953365] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2618.331419] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2618.694098] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2619.046973] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2619.394907] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2619.737661] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2620.092917] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2620.493521] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2620.854286] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2621.231977] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2621.585493] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2621.980344] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2622.357895] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2622.739087] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2623.119238] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2623.496546] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2623.885492] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2624.275502] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2624.667954] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2625.080783] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2625.459308] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2625.875352] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2626.252871] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2626.653951] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2627.039983] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2627.464826] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2627.880787] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2628.271138] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2628.671950] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2629.053648] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2629.482277] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2629.884605] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2630.266695] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2630.638253] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2631.002233] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2631.346036] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2631.706757] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2632.058992] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2632.433706] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2632.784343] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2633.155977] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2633.552576] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2633.911556] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2634.275443] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2634.672814] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2635.062980] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2635.452469] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2635.843490] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2636.241722] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2636.626951] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2637.009048] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2637.396070] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2637.754747] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2638.128326] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2638.500371] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2638.860952] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2639.227949] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2639.592056] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2639.950558] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2640.314213] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2640.661814] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2641.019653] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2641.383502] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2641.731292] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2642.075087] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2642.433747] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2642.771873] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2643.115669] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2643.467108] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2643.819882] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2644.179064] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2644.553648] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2644.903889] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2645.251403] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2645.623706] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2645.981914] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2646.318676] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2646.665628] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2647.020869] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2647.393273] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2647.763336] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2648.161660] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2648.541352] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2648.913118] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2649.296728] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2649.676565] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2650.052730] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2650.420587] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2650.799660] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2651.213528] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2651.607234] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2651.988557] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2652.352093] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2652.726880] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2653.126710] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2653.521248] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2653.928840] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2654.321316] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2654.735226] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2655.135589] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2655.520937] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2655.909885] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2656.285547] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2656.673759] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2657.045101] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2657.427412] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2657.799571] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2658.192817] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2658.549145] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2658.932266] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2659.342109] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2659.800968] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2660.191983] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2660.592716] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2661.002088] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2661.421401] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2661.809950] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2662.183637] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2662.547416] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2662.895597] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2663.237868] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2663.596042] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2663.949740] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2664.295297] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2664.642268] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2665.001850] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2665.345876] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2665.696459] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2666.027526] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2666.396170] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2666.741130] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2667.082041] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2667.417410] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2667.755590] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2668.107691] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2668.477477] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2668.842919] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2669.217252] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2669.562286] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2669.919898] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2670.260127] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2670.626861] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2670.963173] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2671.298034] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2671.639212] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2671.991935] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2672.332158] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2672.684739] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2673.024073] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2673.377996] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2673.730482] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2674.112824] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2674.472204] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2674.826936] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2675.185147] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2675.535973] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2675.908873] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2676.270078] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2676.646614] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2677.033433] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2677.442431] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2677.897236] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2678.299647] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2678.724503] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2679.105945] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2679.481346] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2679.878983] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2680.272174] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2680.672074] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2681.080060] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2681.501109] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2681.899617] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2682.279478] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2682.676290] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2683.058844] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2683.451761] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2683.854409] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2684.244320] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2684.618194] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2684.975485] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2685.310235] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2685.656134] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2686.008655] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2686.353947] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2686.697190] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2687.058797] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2687.414506] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2687.770376] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2688.127091] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2688.486497] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2688.883032] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2689.319635] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2689.793340] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2690.193664] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2690.586018] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2690.951021] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2691.313397] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2691.668762] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2692.049397] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2692.453915] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2692.844919] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2693.252301] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2693.654680] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2694.046940] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2694.432896] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2694.822167] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2695.228294] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2695.603789] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 2695.606960] Lustre: Skipped 340 previous similar messages [ 2695.616214] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2695.621973] Lustre: Skipped 340 previous similar messages [ 2695.624495] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 2695.626415] Lustre: Skipped 340 previous similar messages [ 2695.628795] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2695.637934] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 2695.640815] Lustre: Skipped 340 previous similar messages [ 2696.008248] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2696.387619] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2696.763399] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2697.169341] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2697.543165] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2697.931418] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2698.291743] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2698.694584] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2699.054339] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2699.429033] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2699.781936] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2700.134036] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2700.482053] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2700.866555] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2701.246070] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2701.595660] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2701.975284] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2702.369786] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2702.773575] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2703.159048] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2703.536236] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2703.908519] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2704.272698] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2704.643829] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2705.030322] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2705.417163] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2705.863501] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2706.265482] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2706.671542] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2707.127120] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2707.550089] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2707.937880] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2708.297256] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2708.704947] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2709.096923] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2709.496257] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2709.882485] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2710.299877] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2710.712565] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2711.151259] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2711.596799] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2712.016493] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2712.435854] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2712.865618] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2713.298522] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2713.714026] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2714.146155] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2714.585559] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2715.059542] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2715.518473] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2715.973952] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2716.447061] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2716.881919] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2717.321121] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2717.778131] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2718.219368] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2718.642794] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2719.080313] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2719.539790] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2720.006297] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2720.442973] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2720.859769] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2721.311130] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2721.744521] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2722.147223] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2722.576923] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2722.995317] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2723.424708] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2723.883751] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2724.359731] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2724.858470] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2725.297270] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2725.811483] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2726.301253] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2726.744306] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2727.225621] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2727.670400] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2728.081932] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2728.466516] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2728.876416] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2729.287720] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2729.738802] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2730.181448] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2730.564184] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2730.934604] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2731.313659] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2731.707214] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2732.108891] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2732.542679] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2732.929314] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2733.319248] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2733.687497] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2734.091318] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2734.500347] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2734.905623] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2735.354350] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2735.777341] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2736.230975] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2736.630212] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2737.038647] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2737.446102] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2737.874703] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2738.280566] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2738.729522] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2739.146748] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2739.570877] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2739.981979] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2740.363309] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2740.748830] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2741.149132] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2741.531383] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2741.920400] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2742.333578] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2742.759672] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2743.120786] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2743.527522] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2743.906791] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2744.296925] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2744.673842] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2745.019428] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2745.357974] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2745.700255] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2746.058168] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2746.408193] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2746.753108] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2747.131438] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2747.539390] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2747.923883] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2748.297392] LustreError: 162708:0:(osp_precreate.c:651:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 2748.302818] LustreError: 162708:0:(osp_precreate.c:1355:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 2748.304339] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2748.722081] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2749.109829] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2749.513710] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2749.905916] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2750.332496] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2750.715703] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2751.069663] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2751.435053] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2751.817367] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2752.161043] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2752.511297] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2752.847361] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2753.211543] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2753.584679] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2753.977476] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2754.344565] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2754.692826] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2755.046638] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2755.400198] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2755.751412] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2756.105485] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2756.446476] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2756.810404] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2757.179143] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2757.571721] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2757.946734] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2758.332644] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2758.743543] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2759.147523] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2759.539088] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2759.925685] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2760.284978] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2760.637486] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2761.002070] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2761.369369] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2761.724297] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2762.098590] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2762.459817] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2762.834391] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2763.185141] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2763.535313] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2763.883623] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2764.226518] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2764.563618] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2764.930350] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2765.266389] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2765.654050] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2766.004455] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2766.376315] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2766.763154] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2767.185204] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2767.581875] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2767.953289] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2768.352745] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2768.743895] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2769.170981] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2769.577031] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2769.952076] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2770.331416] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2770.713207] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2771.081409] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2771.465219] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2771.853366] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2772.249062] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2772.652975] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2773.074193] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2773.463941] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2773.871428] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2774.292432] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2774.713176] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2775.112683] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2775.546555] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2775.936289] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2776.323935] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2776.769290] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2777.160341] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2777.565325] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2777.993144] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2778.384481] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2778.759758] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2779.117920] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2779.512289] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2779.897741] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2780.291713] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2780.694876] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2781.108212] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2781.516658] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2781.932548] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2782.322589] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2782.751374] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2783.175185] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2783.576732] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2783.970092] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2784.367040] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2784.757358] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2785.157099] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2785.508844] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2785.866509] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2786.248537] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2786.628228] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2787.019950] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2787.427798] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2787.832728] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2788.233496] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2788.673161] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2789.106027] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2789.522896] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2789.925589] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2790.333456] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2790.749721] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2791.162789] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2791.639066] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2792.032122] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2792.448612] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2792.877274] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2793.280307] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2793.709055] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2794.102975] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2794.511801] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2794.923828] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2795.354517] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2795.745569] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2796.144354] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2796.552327] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2796.926631] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2797.291842] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2797.632604] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2797.991495] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2798.386764] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2798.785269] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2799.181773] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2799.551088] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2799.901925] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2800.307316] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2800.736733] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2801.140352] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2801.547556] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2801.964496] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2802.335635] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2802.735106] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2803.111438] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2803.528323] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2803.970745] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2804.375339] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2804.799645] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2805.160389] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2805.566928] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2805.937087] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2806.317989] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2806.676471] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2807.058448] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2807.436564] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2807.831469] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2808.197709] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2808.578239] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2808.988606] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2809.372811] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2809.738583] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2810.109268] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2810.514753] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2810.912847] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2811.291744] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2811.661386] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2812.040778] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2812.463557] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2812.860761] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2813.259598] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2813.653627] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2814.062761] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2814.447665] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2814.870063] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2815.244162] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2815.660342] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2816.069918] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2816.493901] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2816.919163] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2817.323394] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2817.725755] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2818.135648] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2818.549922] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2818.996234] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2819.387575] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2819.768524] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2820.180519] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2820.611839] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2821.051872] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2821.414746] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2821.789638] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2822.163271] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2822.526516] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2822.867525] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2823.231640] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2823.615584] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2824.001471] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2824.423728] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2824.832997] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2825.228997] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2825.679119] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2826.114356] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2826.537015] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2826.937368] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2827.334731] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2827.751358] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2828.154772] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2985.008792] LustreError: 164283:0:(ofd_dev.c:1780:ofd_create_hdl()) lustre-OST0000: unable to precreate: rc = -28 [ 2985.014453] LustreError: 162708:0:(osp_precreate.c:651:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -28 [ 3115.488244] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3115.490606] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3115.494886] Lustre: Skipped 333 previous similar messages [ 3115.497795] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3116.514543] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3116.516103] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3116.518400] Lustre: Skipped 1 previous similar message [ 3121.632340] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3121.635190] Lustre: Skipped 1 previous similar message [ 3126.752464] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3126.754772] Lustre: Skipped 1 previous similar message [ 3129.311131] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 3129.344701] LustreError: 216476:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 3129.421802] Lustre: server umount lustre-OST0000 complete [ 3131.874810] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3131.877407] Lustre: Skipped 1 previous similar message [ 3142.112239] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3142.114523] Lustre: Skipped 3 previous similar messages [ 3145.183206] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 3145.231700] LustreError: 216680:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup [ 3145.234479] LustreError: 216680:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 3145.319054] Lustre: server umount lustre-MDT0000 complete [ 3146.786431] LustreError: 162669:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1764747726 with bad export cookie 15036387853327872471 [ 3146.789818] LustreError: MGC192.168.201.118@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3146.791706] LustreError: 162669:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 3146.863355] LustreError: 216884:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [ 3146.866072] LustreError: 216884:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 3146.927943] Lustre: server umount lustre-MDT0001 complete [ 3151.074919] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing unload_modules_local [ 3152.270785] Key type lgssc unregistered [ 3152.422665] LNet: 217415:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3152.425746] LNetError: 217415:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3152.434410] LNet: Removed LNI 192.168.201.118@tcp [ 3152.795989] Key type .llcrypt unregistered [ 3152.797723] Key type ._llcrypt unregistered [ 3164.173514] Key type ._llcrypt registered [ 3164.175458] Key type .llcrypt registered [ 3164.223313] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing load_modules_local [ 3164.652178] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3164.720118] alg: No test for adler32 (adler32-zlib) [ 3165.591282] Lustre: Lustre: Build Version: 2.17.0_RC1_2_g29d2772 [ 3165.690241] LNet: Added LNI 192.168.201.118@tcp [8/256/0/180] [ 3167.279196] Key type lgssc registered [ 3167.732416] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3170.677229] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3170.688451] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3171.874960] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3173.219797] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 3176.448211] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3176.592805] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3178.083898] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 3180.100416] Lustre: DEBUG MARKER: oleg118-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 3181.438720] Lustre: DEBUG MARKER: oleg118-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 3183.865234] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3183.980339] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3185.917305] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 3187.930159] Lustre: DEBUG MARKER: oleg118-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3198.310830] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49666 to 0x280000401:49697) [ 3200.529959] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 3201.115835] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3201.119801] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 3201.123686] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3201.127384] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 3201.127404] LustreError: 218805:0:(osp_precreate.c:1263:osp_precreate_thread()) cfs_fail_timeout id 2107 sleeping for 20000ms [ 3203.554252] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3203.556353] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3203.558983] Lustre: Skipped 1 previous similar message [ 3207.916929] LustreError: 221206:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 3207.960765] Lustre: server umount lustre-OST0000 complete [ 3221.207186] LustreError: 218805:0:(osp_precreate.c:1263:osp_precreate_thread()) cfs_fail_timeout id 2107 awake [ 3235.989842] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3236.105727] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3238.119496] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 3240.195287] Lustre: DEBUG MARKER: oleg118-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3243.302685] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3243.310537] LustreError: lustre-OST0000-osc-MDT0001: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3243.310729] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 3243.317832] Lustre: Skipped 1 previous similar message [ 3245.836457] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid in FULL state after 5 sec [ 3247.110348] Lustre: DEBUG MARKER: oleg118-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff981653def800.ost_server_uuid 50 [ 3247.639084] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff981653def800.ost_server_uuid in FULL state after 0 sec [ 3253.730798] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3253.731677] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3253.735427] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3253.738184] Lustre: Skipped 3 previous similar messages [ 3254.699742] LustreError: 222529:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 3254.702163] LustreError: 222529:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 3254.736271] Lustre: server umount lustre-OST0000 complete [ 3258.849057] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3258.849798] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3258.857138] Lustre: Skipped 1 previous similar message [ 3258.859978] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3258.862878] Lustre: Skipped 1 previous similar message [ 3262.012216] LustreError: 222730:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup [ 3262.015294] LustreError: 222730:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 3262.042215] Lustre: server umount lustre-MDT0000 complete [ 3263.274234] LustreError: 221708:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1764747842 with bad export cookie 16402364987431114135 [ 3263.278333] LustreError: MGC192.168.201.118@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3263.278858] LustreError: 221708:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 3263.412448] Lustre: server umount lustre-MDT0001 complete [ 3266.775213] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing unload_modules_local [ 3267.858736] Key type lgssc unregistered [ 3267.991329] LNet: 223462:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3267.993760] LNetError: 223462:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3267.999283] LNet: Removed LNI 192.168.201.118@tcp [ 3268.283425] Key type .llcrypt unregistered [ 3268.284493] Key type ._llcrypt unregistered [ 3279.498391] Key type ._llcrypt registered [ 3279.499877] Key type .llcrypt registered [ 3279.537880] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing load_modules_local [ 3279.821062] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3279.834888] alg: No test for adler32 (adler32-zlib) [ 3280.684113] Lustre: Lustre: Build Version: 2.17.0_RC1_2_g29d2772 [ 3280.770436] LNet: Added LNI 192.168.201.118@tcp [8/256/0/180] [ 3282.351141] Key type lgssc registered [ 3282.713316] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3285.748244] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3285.754431] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3286.915232] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3288.276506] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 3291.074680] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3291.207879] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3292.532180] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 3293.455485] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 3295.580480] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3295.690473] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3297.511308] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 3297.767563] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49699 to 0x280000401:49729) [ 3300.534510] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3300.562411] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3300.621827] Lustre: lustre-OST0001: new disk, initializing [ 3300.623487] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 3300.653754] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 3301.887802] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 3301.891380] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 3301.928043] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 3302.541343] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 3307.833253] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 3315.529549] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 3332.063779] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3332.066579] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3332.070417] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3332.578241] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3332.580018] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3332.581618] Lustre: Skipped 2 previous similar messages [ 3332.583701] Lustre: Skipped 1 previous similar message [ 3333.885206] LustreError: 228507:0:(obd_class.h:479:obd_check_dev()) Device 28 not setup [ 3333.917822] Lustre: server umount lustre-MDT0000 complete [ 3335.181215] LustreError: 228509:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1764747914 with bad export cookie 1386310083470582910 [ 3335.184369] LustreError: MGC192.168.201.118@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3335.185239] LustreError: 228509:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 3335.252082] LustreError: 228710:0:(obd_class.h:479:obd_check_dev()) Device 27 not setup [ 3335.254813] LustreError: 228710:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 3335.305595] Lustre: server umount lustre-MDT0001 complete [ 3345.842118] LustreError: 228911:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 3345.844782] LustreError: 228911:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 3345.864470] Lustre: server umount lustre-OST0000 complete [ 3357.363822] LustreError: 229113:0:(obd_class.h:479:obd_check_dev()) Device 2 not setup [ 3357.365796] LustreError: 229113:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 3357.413381] Lustre: server umount lustre-OST0001 complete [ 3358.576885] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro [ 3358.614889] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3358.998524] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro [ 3359.035175] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3359.438902] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: ,errors=remount-ro [ 3359.485622] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3359.866947] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: ,errors=remount-ro [ 3359.900493] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3364.632670] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing load_modules_local [ 3368.145941] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3368.294397] LustreError: 230520:0:(ldlm_lib.c:1179:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3368.302922] LustreError: 230520:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 3368.320947] Lustre: mylustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3369.692148] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 3372.797374] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3374.352248] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 3375.498202] Lustre: 231627:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 3377.716958] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3377.840271] Lustre: mylustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3377.843768] Lustre: Skipped 1 previous similar message [ 3378.851898] LustreError: 231982:0:(ldlm_lib.c:1179:target_handle_connect()) mylustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3378.857077] Lustre: mylustre-OST0000: new connection from mylustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49731 to 0x280000401:49761) [ 3379.819550] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 3382.842295] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3383.982171] Lustre: mylustre-OST0001: new connection from mylustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:33) [ 3384.920267] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 3388.165370] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 3389.387496] Lustre: 233472:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 3398.623758] LustreError: mylustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3398.626115] Lustre: mylustre-MDT0000-osp-MDT0001: Connection to mylustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3398.630380] Lustre: mylustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3398.632326] Lustre: Skipped 1 previous similar message [ 3404.731786] LustreError: 233778:0:(obd_class.h:479:obd_check_dev()) Device 10 not setup [ 3404.733736] LustreError: 233778:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 3404.758993] Lustre: server umount mylustre-MDT0000 complete [ 3404.769848] LustreError: 230514:0:(ldlm_lib.c:1179:target_handle_connect()) mylustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3404.774717] LustreError: 230514:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 3406.013283] LustreError: 231245:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1764747985 with bad export cookie 1386310083470585675 [ 3406.016859] LustreError: MGC192.168.201.118@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3406.017567] LustreError: 231245:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 3406.131661] Lustre: server umount mylustre-MDT0001 complete [ 3417.081154] LustreError: 234180:0:(obd_class.h:479:obd_check_dev()) Device 23 not setup [ 3417.085044] LustreError: 234180:0:(obd_class.h:479:obd_check_dev()) Skipped 15 previous similar messages [ 3417.108592] Lustre: server umount mylustre-OST0000 complete [ 3428.642420] Lustre: server umount mylustre-OST0001 complete [ 3429.771637] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro [ 3429.799963] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3430.155424] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro [ 3430.182532] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3430.565486] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: ,errors=remount-ro [ 3430.590091] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3430.967750] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: ,errors=remount-ro [ 3431.005893] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3435.791744] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing load_modules_local [ 3439.733778] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3439.872962] LustreError: 235791:0:(ldlm_lib.c:1179:target_handle_connect()) tfs-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3439.896959] Lustre: tfs-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3439.898847] Lustre: Skipped 1 previous similar message [ 3441.132314] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 3444.026799] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3445.452145] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 3446.392828] Lustre: 236902:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 3448.572338] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3450.493626] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 3450.723506] LustreError: 237257:0:(ldlm_lib.c:1179:target_handle_connect()) tfs-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3450.726934] Lustre: tfs-OST0000: new connection from tfs-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49763 to 0x280000401:49793) [ 3453.343932] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3454.440675] Lustre: tfs-OST0001: new connection from tfs-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:65) [ 3455.188862] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 3458.250330] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 3459.352625] Lustre: 238749:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 3469.794866] LustreError: tfs-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3469.794931] Lustre: tfs-MDT0000-lwp-OST0001: Connection to tfs-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3469.796068] Lustre: tfs-MDT0000: Not available for connect from 0@lo (stopping) [ 3469.796074] Lustre: Skipped 4 previous similar messages [ 3469.802479] Lustre: Skipped 6 previous similar messages [ 3472.323566] LustreError: 238999:0:(obd_class.h:479:obd_check_dev()) Device 10 not setup [ 3472.325251] LustreError: 238999:0:(obd_class.h:479:obd_check_dev()) Skipped 9 previous similar messages [ 3472.347833] Lustre: server umount tfs-MDT0000 complete [ 3473.507066] LustreError: 238913:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1764748052 with bad export cookie 1386310083470587565 [ 3473.510166] LustreError: 238913:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 3473.510179] LustreError: MGC192.168.201.118@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3473.626588] Lustre: server umount tfs-MDT0001 complete [ 3485.125150] Lustre: server umount tfs-OST0000 complete [ 3496.671441] Lustre: server umount tfs-OST0001 complete [ 3497.795323] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro [ 3497.821981] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3498.208659] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro [ 3498.238387] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3498.626341] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: ,errors=remount-ro [ 3498.656780] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3499.032727] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: ,errors=remount-ro [ 3499.061598] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3503.911385] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing load_modules_local [ 3507.441446] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3507.597611] LustreError: 241009:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3507.621120] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3507.623048] Lustre: Skipped 3 previous similar messages [ 3509.055359] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 3511.716791] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3513.017314] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 3513.992421] Lustre: 242120:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 3516.341500] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3517.481303] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49763 to 0x280000401:49825) [ 3518.501459] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 3521.519975] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3522.668234] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:97) [ 3523.689501] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 3526.907919] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 3528.119727] Lustre: 243963:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 3535.443508] Lustre: DEBUG MARKER: == conf-sanity test 104a: Make sure user defined options are reflected in mount ========================================================== 02:48:34 (1764748114) [ 3547.615900] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3547.617944] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3547.621946] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3547.623647] Lustre: Skipped 3 previous similar messages [ 3553.280407] LustreError: 244331:0:(obd_class.h:479:obd_check_dev()) Device 10 not setup [ 3553.283280] LustreError: 244331:0:(obd_class.h:479:obd_check_dev()) Skipped 25 previous similar messages [ 3553.309203] Lustre: server umount lustre-MDT0000 complete [ 3553.763374] LustreError: 241004:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3553.770735] LustreError: 241004:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 4 previous similar messages [ 3554.483389] LustreError: 241738:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1764748133 with bad export cookie 1386310083470589231 [ 3554.486774] LustreError: 241738:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 1 previous similar message [ 3554.488962] LustreError: MGC192.168.201.118@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3558.880431] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3558.881451] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 3558.885518] Lustre: Skipped 4 previous similar messages [ 3558.888081] Lustre: Skipped 5 previous similar messages [ 3560.684419] Lustre: server umount lustre-MDT0001 complete [ 3576.287060] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 3576.322464] Lustre: server umount lustre-OST0000 complete [ 3583.637291] Lustre: server umount lustre-OST0001 complete [ 3585.785385] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_hostid [ 3588.891236] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing load_modules_local [ 3592.091380] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3594.128784] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3596.035756] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3598.033024] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3600.292338] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3600.317855] LDISKFS-fs (dm-0): Mount option "noacl" will be removed by 3.5 [ 3600.317855] Contact linux-ldiskfs@vger.kernel.org if you think we should keep it. [ 3600.317855] [ 3600.323327] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: acl,user_xattr,errors=remount-ro,noacl,no_mbcache,nodelalloc [ 3600.410921] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3600.425326] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3600.463923] Lustre: lustre-MDT0000: new disk, initializing [ 3600.488642] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3600.490586] Lustre: Skipped 3 previous similar messages [ 3600.496668] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3601.871967] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 3606.166917] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3606.189220] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3606.223126] Lustre: 247115:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 3606.247927] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 3606.249761] Lustre: Skipped 1 previous similar message [ 3606.284723] Lustre: lustre-MDT0001: new disk, initializing [ 3606.336675] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 3606.341565] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3607.736744] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 3611.904224] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3611.925749] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3612.019291] Lustre: lustre-OST0000: new disk, initializing [ 3612.021309] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3613.300283] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 3613.302912] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 3613.339646] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3613.765283] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 3617.942796] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3617.970376] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3618.040713] Lustre: lustre-OST0001: new disk, initializing [ 3618.043367] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 3619.332323] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 3619.336745] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 3619.382343] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 3620.164140] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 3624.857059] Lustre: DEBUG MARKER: == conf-sanity test 104b: Mount uses last flock argument ========================================================== 02:50:03 (1764748203) [ 3628.724987] Lustre: DEBUG MARKER: == conf-sanity test 105: check file creation for ro and rw bind mnt pt ========================================================== 02:50:07 (1764748207) [ 3644.384246] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3644.384361] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3644.386461] LustreError: Skipped 1 previous similar message [ 3644.390373] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3644.394357] Lustre: Skipped 3 previous similar messages [ 3646.125868] LustreError: 250539:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 3646.127752] LustreError: 250539:0:(obd_class.h:479:obd_check_dev()) Skipped 25 previous similar messages [ 3646.150194] Lustre: server umount lustre-OST0000 complete [ 3650.017770] LustreError: 248912:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3650.023902] LustreError: 248912:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 4 previous similar messages [ 3653.727737] Lustre: server umount lustre-MDT0000 complete [ 3655.074274] LustreError: 247105:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1764748234 with bad export cookie 1386310083470590785 [ 3655.078578] LustreError: MGC192.168.201.118@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3655.078746] LustreError: 247105:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 3661.232216] Lustre: server umount lustre-MDT0001 complete [ 3665.000761] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing unload_modules_local [ 3674.719772] Lustre: server umount lustre-OST0001 complete [ 3676.831653] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_hostid [ 3683.331599] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing load_modules_local [ 3687.047371] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3689.316658] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3691.465661] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3693.814249] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3696.202331] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3696.224154] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3696.305859] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3696.319747] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3696.359347] Lustre: lustre-MDT0000: new disk, initializing [ 3696.401142] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3697.770596] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 3702.238196] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3702.267429] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3702.309704] Lustre: 253999:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 3702.425798] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3703.856723] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 3707.035602] Lustre: DEBUG MARKER: oleg118-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 3708.214419] Lustre: DEBUG MARKER: oleg118-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 3710.475467] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3710.499813] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3712.377447] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 3712.414026] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3712.453409] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 3715.424488] Lustre: DEBUG MARKER: oleg118-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3722.720954] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3722.722644] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3722.724409] LustreError: Skipped 2 previous similar messages [ 3722.727380] Lustre: Skipped 5 previous similar messages [ 3722.728118] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3722.733104] Lustre: Skipped 7 previous similar messages [ 3723.727296] Lustre: server umount lustre-OST0000 complete [ 3731.164605] Lustre: server umount lustre-MDT0000 complete [ 3732.477422] LustreError: 254779:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1764748311 with bad export cookie 1386310083470592255 [ 3732.480482] LustreError: MGC192.168.201.118@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3732.483129] LustreError: 254779:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 3732.614413] Lustre: server umount lustre-MDT0001 complete [ 3736.514615] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing unload_modules_local [ 3737.532766] Key type lgssc unregistered [ 3737.660594] LNet: 257373:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3737.663455] LNetError: 257373:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3737.670516] LNet: Removed LNI 192.168.201.118@tcp [ 3737.984162] Key type .llcrypt unregistered [ 3737.985841] Key type ._llcrypt unregistered [ 3747.425596] Key type ._llcrypt registered [ 3747.426771] Key type .llcrypt registered [ 3747.463753] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing load_modules_local [ 3747.818473] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3747.852552] alg: No test for adler32 (adler32-zlib) [ 3748.707853] Lustre: Lustre: Build Version: 2.17.0_RC1_2_g29d2772 [ 3748.805610] LNet: Added LNI 192.168.201.118@tcp [8/256/0/180] [ 3750.391182] Key type lgssc registered [ 3750.761403] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3753.333709] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3753.340647] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3754.484730] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3755.850437] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 3758.689987] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3758.813990] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3760.163180] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 3761.920339] Lustre: DEBUG MARKER: oleg118-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 3763.137687] Lustre: DEBUG MARKER: oleg118-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 3765.387055] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3765.491934] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3767.373940] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 3769.072572] Lustre: DEBUG MARKER: oleg118-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3769.383178] Lustre: Setting parameter lustre-OST0000.ost.unknown_param=50 in log lustre-OST0000 [ 3774.945312] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3774.945334] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3774.950115] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3776.044079] LustreError: 261083:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 3776.083599] Lustre: server umount lustre-OST0000 complete [ 3777.443942] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3777.451747] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3777.455213] Lustre: Skipped 1 previous similar message [ 3782.624594] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3782.628136] Lustre: Skipped 1 previous similar message [ 3783.554553] LustreError: 261293:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup [ 3783.557872] LustreError: 261293:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 3783.587535] Lustre: server umount lustre-MDT0000 complete [ 3784.910577] LustreError: 260983:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1764748364 with bad export cookie 16442272624619387964 [ 3784.916265] LustreError: MGC192.168.201.118@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3784.984156] LustreError: 261496:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [ 3784.986109] LustreError: 261496:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 3785.050408] Lustre: server umount lustre-MDT0001 complete [ 3788.785738] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing unload_modules_local [ 3789.913852] Key type lgssc unregistered [ 3790.043382] LNet: 262027:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3790.045530] LNetError: 262027:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3790.055293] LNet: Removed LNI 192.168.201.118@tcp [ 3790.385152] Key type .llcrypt unregistered [ 3790.386345] Key type ._llcrypt unregistered [ 3797.398952] Key type ._llcrypt registered [ 3797.399978] Key type .llcrypt registered [ 3797.434879] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing load_modules_local [ 3797.730717] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3797.747098] alg: No test for adler32 (adler32-zlib) [ 3798.598798] Lustre: Lustre: Build Version: 2.17.0_RC1_2_g29d2772 [ 3798.669487] LNet: Added LNI 192.168.201.118@tcp [8/256/0/180] [ 3800.247139] Key type lgssc registered [ 3800.590594] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3803.519517] Lustre: lustre-OST0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3803.527293] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3818.975387] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 3824.095270] LustreError: 263176:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.201.118@tcp: failed processing log, type 4: rc = -110 [ 3850.719169] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3852.847929] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 3854.707579] Lustre: DEBUG MARKER: oleg118-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3855.471879] LustreError: 263841:0:(obd_class.h:479:obd_check_dev()) Device 1 not setup [ 3855.513662] Lustre: server umount lustre-OST0000 complete [ 3860.650362] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing unload_modules_local [ 3861.629120] Key type lgssc unregistered [ 3861.757443] LNet: 264677:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3861.761233] LNetError: 264677:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3861.769303] LNet: Removed LNI 192.168.201.118@tcp [ 3862.050610] Key type .llcrypt unregistered [ 3862.051807] Key type ._llcrypt unregistered [ 3875.502265] Key type ._llcrypt registered [ 3875.503293] Key type .llcrypt registered [ 3875.541701] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing load_modules_local [ 3875.925467] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3875.983084] alg: No test for adler32 (adler32-zlib) [ 3876.850592] Lustre: Lustre: Build Version: 2.17.0_RC1_2_g29d2772 [ 3876.947934] LNet: Added LNI 192.168.201.118@tcp [8/256/0/180] [ 3878.543213] Key type lgssc registered [ 3878.913755] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3883.461037] loop0: detected capacity change from 0 to 204800000 [ 3883.548617] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 3883.552651] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 3883.690706] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 3883.712050] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3884.085529] loop0: detected capacity change from 0 to 204800000 [ 3884.139891] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 3884.143668] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 3884.281675] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 3884.302090] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3884.647479] loop0: detected capacity change from 0 to 204800000 [ 3884.717015] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 3884.721649] blk_update_request: operation not supported error, dev loop0, sector 8224 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 3884.764834] blk_update_request: operation not supported error, dev loop0, sector 58264 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 3884.818256] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3885.186902] loop0: detected capacity change from 0 to 204800000 [ 3885.247481] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 3885.344197] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3885.708524] loop0: detected capacity change from 0 to 204800000 [ 3885.719595] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 3887.146113] loop0: detected capacity change from 0 to 204800000 [ 3887.158298] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 3888.519077] loop0: detected capacity change from 0 to 204800000 [ 3888.528131] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 3889.902250] loop0: detected capacity change from 0 to 204800000 [ 3889.911521] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 3891.286083] loop0: detected capacity change from 0 to 204800000 [ 3891.319568] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3891.355526] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3891.362778] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3891.388981] Lustre: 268050:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 3891.392873] Lustre: 268050:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000003:0xb:0x0]: rc = 0 [ 3891.399579] Lustre: 268050:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000003:0xa:0x0]: rc = 0 [ 3891.403249] Lustre: 268050:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x10000-MDT0000' with [0x200000005:0x1:0x0]: rc = 0 [ 3891.407827] Lustre: 268050:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000003:0xc:0x0]: rc = 0 [ 3891.413061] Lustre: 268050:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x2010000-MDT0000' with [0x200000005:0x3:0x0]: rc = 0 [ 3891.417482] Lustre: 268050:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x1010000-MDT0000' with [0x200000005:0x2:0x0]: rc = 0 [ 3891.423259] Lustre: 268050:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index 'nodemap' with [0x200000003:0x3:0x0]: rc = 0 [ 3891.427732] Lustre: 268050:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x1020000' with [0x200000006:0x1020000:0x0]: rc = 0 [ 3891.432394] Lustre: 268050:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x20000' with [0x200000006:0x20000:0x0]: rc = 0 [ 3891.435936] Lustre: 268050:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x2020000' with [0x200000006:0x2020000:0x0]: rc = 0 [ 3891.440928] Lustre: 268050:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000006:0x1010000:0x0]: rc = 0 [ 3891.447307] Lustre: 268050:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000006:0x10000:0x0]: rc = 0 [ 3891.453407] Lustre: 268050:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000006:0x2010000:0x0]: rc = 0 [ 3894.009353] LustreError: 268315:0:(obd_class.h:479:obd_check_dev()) Device 2 not setup [ 3894.053952] Lustre: server umount MGS complete [ 3894.366891] loop0: detected capacity change from 0 to 204800000 [ 3894.402321] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3894.570678] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3894.588648] Lustre: lustre-MDT0000: Aborting client recovery [ 3894.926079] loop1: detected capacity change from 0 to 204800000 [ 3894.995842] LDISKFS-fs (loop1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3895.026397] LDISKFS-fs (loop1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3895.045834] Lustre: 268484:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 3895.049480] Lustre: 268484:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0001: restore index '0x1010000-MDT0001' with [0x200000005:0x2:0x0]: rc = 0 [ 3895.053581] Lustre: 268484:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0001: restore index '0x2010000-MDT0001' with [0x200000005:0x3:0x0]: rc = 0 [ 3895.057293] Lustre: 268484:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0001: restore index '0x2010000' with [0x200000003:0x6:0x0]: rc = 0 [ 3895.061789] Lustre: 268484:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0001: restore index '0x1010000' with [0x200000003:0x5:0x0]: rc = 0 [ 3895.066508] Lustre: 268484:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0001: restore index '0x10000' with [0x200000003:0x4:0x0]: rc = 0 [ 3895.071124] Lustre: 268484:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0001: restore index '0x10000-MDT0001' with [0x200000005:0x1:0x0]: rc = 0 [ 3895.080630] Lustre: Found index 1 for lustre-MDT0001, updating log [ 3895.156646] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3895.176614] Lustre: lustre-MDT0001: Aborting client recovery [ 3895.524277] loop2: detected capacity change from 0 to 204800000 [ 3895.563882] LDISKFS-fs (loop2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3895.604344] LDISKFS-fs (loop2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3895.626055] Lustre: 268569:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 3895.631330] Lustre: 268569:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0000: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 3895.634648] Lustre: 268569:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0000: restore index '0x1020000-OST0000' with [0x200000005:0x2:0x0]: rc = 0 [ 3895.638914] Lustre: 268569:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0000: restore index '0x20000-OST0000' with [0x200000005:0x1:0x0]: rc = 0 [ 3895.643013] Lustre: 268569:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0000: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 3895.648108] Lustre: 268569:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0000: restore index '0x2020000-OST0000' with [0x200000005:0x3:0x0]: rc = 0 [ 3895.652245] Lustre: 268569:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0000: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 3895.703804] Lustre: Found index 0 for lustre-OST0000, updating log [ 3895.741091] Lustre: lustre-OST0000: Aborting recovery [ 3896.067746] loop3: detected capacity change from 0 to 204800000 [ 3896.109181] LDISKFS-fs (loop3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3896.145430] LDISKFS-fs (loop3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3896.164384] Lustre: 268657:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 3896.167832] Lustre: 268657:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0001: restore index '0x20000-OST0001' with [0x200000005:0x1:0x0]: rc = 0 [ 3896.171278] Lustre: 268657:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0001: restore index '0x1020000-OST0001' with [0x200000005:0x2:0x0]: rc = 0 [ 3896.174263] Lustre: 268657:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0001: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 3896.177220] Lustre: 268657:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0001: restore index '0x2020000-OST0001' with [0x200000005:0x3:0x0]: rc = 0 [ 3896.180803] Lustre: 268657:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0001: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 3896.183774] Lustre: 268657:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0001: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 3896.216908] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 3896.218878] Lustre: Skipped 1 previous similar message [ 3901.417599] LustreError: 268427:0:(lod_dev.c:510:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 7, retries 0, failed: rc = -108 [ 3904.482127] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 3904.482975] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 3904.493166] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000bd0 [ 3904.495918] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000bd0 [ 3905.506756] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:5 to 0x280000400:33) [ 3905.506874] Lustre: lustre-OST0001: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:5 to 0x2c0000400:33) [ 3906.673249] LustreError: 268395:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0000: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0000 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0000/identity_upcall, time 256us: rc = -2 [ 3920.864633] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3920.865138] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3920.865369] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3920.868103] Lustre: Skipped 2 previous similar messages [ 3924.035200] LustreError: 269014:0:(obd_class.h:479:obd_check_dev()) Device 10 not setup [ 3924.037162] LustreError: 269014:0:(obd_class.h:479:obd_check_dev()) Skipped 1 previous similar message [ 3924.071668] Lustre: server umount lustre-MDT0000 complete [ 3924.375642] LustreError: 268376:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1764748503 with bad export cookie 9421629403555561433 [ 3924.377919] LustreError: MGC192.168.201.118@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3924.378702] LustreError: 268376:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 3925.983931] LustreError: 268673:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3925.984300] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3925.984575] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 3925.984581] Lustre: Skipped 3 previous similar messages [ 3925.988916] LustreError: 268673:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 3925.997686] Lustre: Skipped 2 previous similar messages [ 3930.015600] LustreError: 268673:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3930.015749] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 3930.023302] LustreError: 268673:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 3930.025144] Lustre: Skipped 2 previous similar messages [ 3930.558377] LustreError: 269066:0:(obd_class.h:479:obd_check_dev()) Device 18 not setup [ 3930.561377] LustreError: 269066:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 3930.616162] Lustre: server umount lustre-MDT0001 complete [ 3937.004094] LustreError: 269117:0:(obd_class.h:479:obd_check_dev()) Device 23 not setup [ 3937.006680] LustreError: 269117:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 3937.034930] Lustre: server umount lustre-OST0000 complete [ 3943.404740] LustreError: 269168:0:(obd_class.h:479:obd_check_dev()) Device 2 not setup [ 3943.407040] LustreError: 269168:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 3943.473462] Lustre: server umount lustre-OST0001 complete [ 3946.126273] Lustre: DEBUG MARKER: == conf-sanity test 109a: test lctl clear_conf fsname ==== 02:55:24 (1764748524) [ 3955.436450] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_hostid [ 3958.781645] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing load_modules_local [ 3962.381779] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3964.651800] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3966.793878] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3968.960886] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3971.300365] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3971.322251] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3971.405359] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3971.421273] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3971.461620] Lustre: lustre-MDT0000: new disk, initializing [ 3971.489973] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3971.498281] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3972.762347] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 3976.917901] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3976.939140] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3976.983971] Lustre: 272712:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 3977.016476] Lustre: lustre-MDT0001: Not available for connect from 0@lo (not set up) [ 3977.017058] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 3977.020355] Lustre: Skipped 1 previous similar message [ 3977.055958] Lustre: lustre-MDT0001: new disk, initializing [ 3977.094162] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3977.107613] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 3977.110951] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3978.453466] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 3981.546591] Lustre: DEBUG MARKER: oleg118-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 3982.758097] Lustre: DEBUG MARKER: oleg118-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 3985.055765] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3985.076778] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3985.179313] Lustre: lustre-OST0000: new disk, initializing [ 3985.181277] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3985.207595] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3987.210896] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 3989.496686] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 3989.499895] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 3989.533764] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3990.389397] Lustre: DEBUG MARKER: oleg118-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3992.821379] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [ 4000.574179] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [ 4008.964629] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [ 4014.466134] Lustre: Modifying parameter lustre-client.llite.max_read_ahead_mb=64 in log lustre-client [ 4066.274981] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4066.278369] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4066.281730] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4066.294574] Lustre: Skipped 1 previous similar message [ 4066.307621] Lustre: Skipped 1 previous similar message [ 4067.959106] LustreError: 276471:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 4067.963144] LustreError: 276471:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 4068.089359] Lustre: server umount lustre-OST0000 complete [ 4071.396396] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4071.410485] Lustre: Skipped 1 previous similar message [ 4076.512995] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4076.518099] Lustre: Skipped 3 previous similar messages [ 4085.215135] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 4085.323712] LustreError: 276672:0:(obd_class.h:479:obd_check_dev()) Device 22 not setup [ 4085.328287] LustreError: 276672:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 4085.443528] Lustre: server umount lustre-MDT0000 complete [ 4086.758036] LustreError: 276104:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4086.778125] LustreError: 276104:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 4088.507153] LustreError: 272702:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1764748667 with bad export cookie 9421629403555564135 [ 4088.509930] LustreError: MGC192.168.201.118@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4088.522299] LustreError: 272702:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4088.762653] Lustre: server umount lustre-MDT0001 complete [ 4094.010632] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4095.739381] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 4097.678092] Lustre: server umount MGS complete [ 4101.894522] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4102.091634] LustreError: 278438:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4102.101609] LustreError: 278450:0:(qmt_pool.c:1523:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [ 4102.133473] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4103.704202] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 4107.013653] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4108.691425] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 4110.771189] Lustre: DEBUG MARKER: oleg118-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 4112.127730] Lustre: DEBUG MARKER: oleg118-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 4114.898107] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4117.203233] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 4119.254753] Lustre: DEBUG MARKER: oleg118-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4148.193189] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4148.196495] LustreError: Skipped 1 previous similar message [ 4148.198730] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4148.204746] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4148.206663] Lustre: Skipped 3 previous similar messages [ 4153.907102] LustreError: 281118:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 4153.909895] LustreError: 281118:0:(obd_class.h:479:obd_check_dev()) Skipped 15 previous similar messages [ 4153.961915] Lustre: server umount lustre-OST0000 complete [ 4158.432920] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4158.438666] Lustre: Skipped 2 previous similar messages [ 4158.441133] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4158.444121] LustreError: Skipped 1 previous similar message [ 4161.501819] Lustre: server umount lustre-MDT0000 complete [ 4162.855204] LustreError: 278418:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1764748742 with bad export cookie 9421629403555565983 [ 4162.859163] LustreError: MGC192.168.201.118@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4162.859289] LustreError: 278418:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4162.975855] Lustre: server umount lustre-MDT0001 complete [ 4166.909163] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing unload_modules_local [ 4167.988492] Key type lgssc unregistered [ 4168.141433] LNet: 282050:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4168.144313] LNetError: 282050:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4168.151334] LNet: Removed LNI 192.168.201.118@tcp [ 4168.459499] Key type .llcrypt unregistered [ 4168.460760] Key type ._llcrypt unregistered [ 4181.015256] Key type ._llcrypt registered [ 4181.016533] Key type .llcrypt registered [ 4181.059335] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_hostid [ 4187.861027] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing load_modules_local [ 4188.261514] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4188.297193] alg: No test for adler32 (adler32-zlib) [ 4189.160055] Lustre: Lustre: Build Version: 2.17.0_RC1_2_g29d2772 [ 4189.251775] LNet: Added LNI 192.168.201.118@tcp [8/256/0/180] [ 4190.840157] Key type lgssc registered [ 4191.356586] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4195.388621] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4198.407600] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4200.817141] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4203.364913] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4206.123381] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4206.145036] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4206.152725] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4207.265180] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 4207.281876] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 4207.323660] Lustre: lustre-MDT0000: new disk, initializing [ 4207.381904] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4207.394444] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 4208.991747] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 4213.770338] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4213.799021] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4213.839686] Lustre: 285962:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 4213.884583] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 4213.888053] Lustre: Skipped 1 previous similar message [ 4213.928040] Lustre: lustre-MDT0001: new disk, initializing [ 4213.960624] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4213.974751] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 4213.978309] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 4215.467155] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 4218.786428] Lustre: DEBUG MARKER: oleg118-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 4220.192815] Lustre: DEBUG MARKER: oleg118-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 4222.657395] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4222.684963] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4222.802875] Lustre: lustre-OST0000: new disk, initializing [ 4222.805792] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 4222.843743] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4223.501122] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 4223.506923] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 4223.559327] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 4225.278643] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 4228.589635] Lustre: DEBUG MARKER: oleg118-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4232.097067] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [ 4234.882168] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [ 4242.411092] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [ 4247.041801] Lustre: Modifying parameter lustre-client.llite.max_read_ahead_mb=64 in log lustre-client [ 4290.019261] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4290.021164] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4290.023108] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4290.026453] Lustre: Skipped 1 previous similar message [ 4291.310093] LustreError: 289488:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 4291.346901] Lustre: server umount lustre-OST0000 complete [ 4295.136623] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4295.137108] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4295.140786] Lustre: Skipped 1 previous similar message [ 4295.143564] Lustre: Skipped 1 previous similar message [ 4298.751769] LustreError: 289689:0:(obd_class.h:479:obd_check_dev()) Device 22 not setup [ 4298.754927] LustreError: 289689:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 4298.786446] Lustre: server umount lustre-MDT0000 complete [ 4300.091442] LustreError: 286742:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1764748879 with bad export cookie 3608059286862482839 [ 4300.094370] LustreError: MGC192.168.201.118@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4300.095662] LustreError: 286742:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4300.162236] LustreError: 289890:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 4300.164529] LustreError: 289890:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 4300.222990] Lustre: server umount lustre-MDT0001 complete [ 4303.246321] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4304.640784] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 4306.374886] LustreError: 290830:0:(obd_class.h:479:obd_check_dev()) Device 2 not setup [ 4306.378019] LustreError: 290830:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 4306.403956] Lustre: server umount MGS complete [ 4310.340040] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4310.518899] LustreError: 291452:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4310.524785] LustreError: 291452:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 4310.524917] LustreError: 291464:0:(qmt_pool.c:1523:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [ 4310.548047] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4311.924827] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 4314.929676] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4315.058743] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4316.449231] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 4318.412353] Lustre: DEBUG MARKER: oleg118-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 4319.660866] Lustre: DEBUG MARKER: oleg118-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 4322.186794] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4324.577946] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 4326.543198] Lustre: DEBUG MARKER: oleg118-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4361.185842] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4361.188205] LustreError: Skipped 1 previous similar message [ 4361.190356] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4361.191828] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4361.194044] Lustre: Skipped 1 previous similar message [ 4365.360989] LustreError: 294134:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 4365.363203] LustreError: 294134:0:(obd_class.h:479:obd_check_dev()) Skipped 1 previous similar message [ 4365.414303] Lustre: server umount lustre-OST0000 complete [ 4371.426459] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4371.426476] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4371.426488] LustreError: Skipped 1 previous similar message [ 4371.427174] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4371.427178] Lustre: Skipped 1 previous similar message [ 4371.431665] Lustre: Skipped 1 previous similar message [ 4376.544428] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4376.546762] Lustre: Skipped 2 previous similar messages [ 4381.151126] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 4381.184819] LustreError: 294335:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup [ 4381.186806] LustreError: 294335:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 4381.214868] Lustre: server umount lustre-MDT0000 complete [ 4381.663938] LustreError: 291452:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4381.670198] LustreError: 291452:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 4382.598328] LustreError: 291434:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1764748961 with bad export cookie 3608059286862484638 [ 4382.602280] LustreError: MGC192.168.201.118@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4382.602965] LustreError: 291434:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4382.726806] Lustre: server umount lustre-MDT0001 complete [ 4386.660640] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing unload_modules_local [ 4387.818953] Key type lgssc unregistered [ 4387.960547] LNet: 295069:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4387.963670] LNetError: 295069:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4387.973608] LNet: Removed LNI 192.168.201.118@tcp [ 4388.335337] Key type .llcrypt unregistered [ 4388.336888] Key type ._llcrypt unregistered [ 4395.988326] Key type ._llcrypt registered [ 4395.989914] Key type .llcrypt registered [ 4396.037782] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_hostid [ 4402.570869] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing load_modules_local [ 4402.995699] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4403.058988] alg: No test for adler32 (adler32-zlib) [ 4403.946571] Lustre: Lustre: Build Version: 2.17.0_RC1_2_g29d2772 [ 4404.049086] LNet: Added LNI 192.168.201.118@tcp [8/256/0/180] [ 4405.639260] Key type lgssc registered [ 4406.098919] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4409.354403] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4411.604406] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4414.902366] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing load_modules_local [ 4418.073349] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4420.502832] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4420.527330] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4420.534133] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4421.627214] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 4421.641304] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 4421.678717] Lustre: lustre-MDT0000: new disk, initializing [ 4421.711660] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4421.721376] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 4423.138765] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 4427.308532] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4427.333458] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4427.421764] Lustre: lustre-OST0000: new disk, initializing [ 4427.423573] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 4427.425147] Lustre: Skipped 1 previous similar message [ 4427.450291] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4429.236167] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 4430.334178] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 4430.338597] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 4430.349082] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 4653.573762] LustreError: 300004:0:(obd_class.h:479:obd_check_dev()) Device 13 not setup [ 4653.718662] Lustre: server umount lustre-MDT0000 complete [ 4655.136625] LustreError: 299049:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1764749234 with bad export cookie 16438424313693969699 [ 4655.141196] LustreError: MGC192.168.201.118@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4665.331417] LustreError: 300206:0:(obd_class.h:479:obd_check_dev()) Device 2 not setup [ 4665.333882] LustreError: 300206:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 4665.388374] Lustre: server umount lustre-OST0000 complete [ 4675.569278] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_hostid [ 4679.578476] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing load_modules_local [ 4684.500550] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4687.576513] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4690.238162] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4692.766250] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4695.587846] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4695.617089] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4695.705960] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 4695.720436] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 4695.758607] Lustre: lustre-MDT0000: new disk, initializing [ 4695.793245] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4695.801707] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 4697.247201] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 4701.652550] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4701.674926] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4701.713074] Lustre: 303247:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 4701.742327] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 4701.745047] Lustre: Skipped 1 previous similar message [ 4701.783900] Lustre: lustre-MDT0001: new disk, initializing [ 4701.819834] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4701.832603] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 4701.836752] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 4703.223506] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 4706.695234] Lustre: DEBUG MARKER: oleg118-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 4708.134528] Lustre: DEBUG MARKER: oleg118-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 4710.893571] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4710.921619] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4711.025938] Lustre: lustre-OST0000: new disk, initializing [ 4711.028368] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 4711.064195] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4713.350298] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 4716.546669] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 4716.551354] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 4716.595226] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 4716.828663] Lustre: DEBUG MARKER: oleg118-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4718.482858] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 4718.569689] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 4720.135855] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 4720.235745] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 4721.632647] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4721.634566] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4721.638305] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4721.641402] Lustre: Skipped 1 previous similar message [ 4722.989184] LustreError: 306187:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 4722.992513] LustreError: 306187:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 4723.041404] Lustre: server umount lustre-OST0000 complete [ 4726.752583] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4726.753925] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4726.758637] Lustre: Skipped 1 previous similar message [ 4726.760688] Lustre: Skipped 1 previous similar message [ 4730.628752] LustreError: 306388:0:(obd_class.h:479:obd_check_dev()) Device 22 not setup [ 4730.632220] LustreError: 306388:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 4730.667810] Lustre: server umount lustre-MDT0000 complete [ 4732.036419] LustreError: 303239:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1764749311 with bad export cookie 16438424313696910476 [ 4732.039496] LustreError: MGC192.168.201.118@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4732.041468] LustreError: 303239:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4732.174236] Lustre: server umount lustre-MDT0001 complete [ 4733.459537] Lustre: DEBUG MARKER: SKIP: conf-sanity test_111 skipping SLOW test 111 [ 4734.141421] Lustre: DEBUG MARKER: == conf-sanity test 112a: mount OST with no_create option ========================================================== 03:08:32 (1764749312) [ 4736.828908] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4737.018055] LustreError: 307161:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4737.024537] LustreError: 307161:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 4737.043566] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4738.623125] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 4741.971186] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4743.722318] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 4745.795228] Lustre: DEBUG MARKER: oleg118-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 4747.186730] Lustre: DEBUG MARKER: oleg118-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 4749.764195] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4751.949722] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 4753.888820] Lustre: DEBUG MARKER: oleg118-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4756.615359] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4756.651871] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4756.723191] Lustre: lustre-OST0001: new disk, initializing [ 4756.725790] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 4756.761883] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 4756.765049] Lustre: Skipped 2 previous similar messages [ 4759.078981] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 4761.099736] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 4761.103969] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 4761.123510] LustreError: 310138:0:(osp_precreate.c:969:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [ 4762.151340] LustreError: 310142:0:(osp_precreate.c:969:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [ 4762.160958] LustreError: 310142:0:(osp_precreate.c:969:osp_precreate_cleanup_orphans()) Skipped 2 previous similar messages [ 4763.172543] LustreError: 310138:0:(osp_precreate.c:969:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [ 4763.177850] LustreError: 310138:0:(osp_precreate.c:969:osp_precreate_cleanup_orphans()) Skipped 1 previous similar message [ 4763.327212] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 4763.428253] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 4764.866917] Lustre: DEBUG MARKER: oleg118-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff98164378d800.ost_server_uuid 50 [ 4765.218477] LustreError: 310138:0:(osp_precreate.c:969:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [ 4765.222282] LustreError: 310138:0:(osp_precreate.c:969:osp_precreate_cleanup_orphans()) Skipped 3 previous similar messages [ 4765.425813] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff98164378d800.ost_server_uuid in FULL state after 0 sec [ 4766.880174] Lustre: DEBUG MARKER: oleg118-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff98164378d800.ost_server_uuid 50 [ 4767.507997] Lustre: DEBUG MARKER: osc.lustre-OST0001-osc-ffff98164378d800.ost_server_uuid in FULL state after 0 sec [ 4768.298651] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 4782.220676] Lustre: lustre-OST0001: Not available for connect from 192.168.201.18@tcp (stopping) [ 4782.223439] Lustre: Skipped 1 previous similar message [ 4782.562731] Lustre: lustre-OST0001-osc-MDT0001: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4782.567991] Lustre: Skipped 1 previous similar message [ 4785.582097] LustreError: 310969:0:(obd_class.h:479:obd_check_dev()) Device 25 not setup [ 4785.584549] LustreError: 310969:0:(obd_class.h:479:obd_check_dev()) Skipped 13 previous similar messages [ 4785.605746] Lustre: server umount lustre-OST0001 complete [ 4787.681499] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4787.682529] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4787.682821] LustreError: 308758:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4787.683693] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4787.683698] Lustre: Skipped 2 previous similar messages [ 4787.685234] Lustre: Skipped 1 previous similar message [ 4787.697057] LustreError: Skipped 1 previous similar message [ 4792.800351] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4792.803081] Lustre: Skipped 2 previous similar messages [ 4792.803510] LustreError: 308758:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4792.811458] LustreError: 308758:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 4793.362723] Lustre: server umount lustre-OST0000 complete [ 4797.920706] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4797.924590] Lustre: Skipped 1 previous similar message [ 4800.767685] LustreError: 311374:0:(obd_class.h:479:obd_check_dev()) Device 28 not setup [ 4800.770644] LustreError: 311374:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 4800.800617] Lustre: server umount lustre-MDT0000 complete [ 4802.146502] LustreError: 307142:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1764749381 with bad export cookie 16438424313696911344 [ 4802.150071] LustreError: MGC192.168.201.118@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4802.151719] LustreError: 307142:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4802.276582] Lustre: server umount lustre-MDT0001 complete [ 4806.188966] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing unload_modules_local [ 4807.310149] Key type lgssc unregistered [ 4807.450503] LNet: 312106:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4807.453263] LNetError: 312106:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4807.461433] LNet: Removed LNI 192.168.201.118@tcp [ 4807.797129] Key type .llcrypt unregistered [ 4807.798546] Key type ._llcrypt unregistered [ 4817.460360] Key type ._llcrypt registered [ 4817.462105] Key type .llcrypt registered [ 4817.502661] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing load_modules_local [ 4817.924190] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4817.963200] alg: No test for adler32 (adler32-zlib) [ 4818.813963] Lustre: Lustre: Build Version: 2.17.0_RC1_2_g29d2772 [ 4818.901396] LNet: Added LNI 192.168.201.118@tcp [8/256/0/180] [ 4820.495203] Key type lgssc registered [ 4820.900102] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4823.789101] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4823.799861] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4825.002160] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4826.570354] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 4828.604226] Lustre: DEBUG MARKER: oleg118-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 4831.098389] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4831.229843] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4832.641932] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 4835.981617] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4836.125206] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4838.263781] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 4840.244071] Lustre: DEBUG MARKER: oleg118-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4841.254103] LustreError: 315038:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4841.261525] LustreError: 315038:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 4842.819765] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4842.911933] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 4845.145544] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 4847.311515] Lustre: DEBUG MARKER: oleg118-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid [ 4850.153702] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:65) [ 4850.154340] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:33) [ 4854.304036] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing wait_import_state (FULL|IDLE) os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [ 4854.387288] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 4932.065233] LustreError: lustre-OST0001-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4932.066409] Lustre: lustre-OST0001-osc-MDT0000: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4932.075167] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 4937.186232] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 4937.188892] Lustre: Skipped 2 previous similar messages [ 4937.260082] LustreError: 316906:0:(obd_class.h:479:obd_check_dev()) Device 27 not setup [ 4937.275725] Lustre: server umount lustre-OST0001 complete [ 4942.304722] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4942.307958] LustreError: 315036:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4942.308064] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4942.309563] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4942.313596] LustreError: 315036:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 4942.318036] Lustre: Skipped 2 previous similar messages [ 4947.426120] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4947.426203] LustreError: 315701:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4947.428594] Lustre: Skipped 2 previous similar messages [ 4947.435263] LustreError: 315701:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 4952.547044] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4952.547066] LustreError: 315701:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4952.549481] Lustre: Skipped 1 previous similar message [ 4952.554329] LustreError: 315701:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 4953.055220] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 4953.075439] LustreError: 317108:0:(obd_class.h:479:obd_check_dev()) Device 23 not setup [ 4953.078720] LustreError: 317108:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 4953.122585] Lustre: server umount lustre-OST0000 complete [ 4957.664529] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4957.668795] Lustre: Skipped 1 previous similar message [ 4960.518603] LustreError: 317310:0:(obd_class.h:479:obd_check_dev()) Device 10 not setup [ 4960.521573] LustreError: 317310:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 4960.556978] Lustre: server umount lustre-MDT0000 complete [ 4961.902541] LustreError: 313465:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1764749541 with bad export cookie 4530973718808872774 [ 4961.904664] LustreError: MGC192.168.201.118@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4961.906765] LustreError: 313465:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 4962.061322] Lustre: server umount lustre-MDT0001 complete [ 4965.681705] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing unload_modules_local [ 4966.722050] Key type lgssc unregistered [ 4966.843583] LNet: 318042:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4966.846485] LNetError: 318042:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4966.856396] LNet: Removed LNI 192.168.201.118@tcp [ 4967.164588] Key type .llcrypt unregistered [ 4967.165948] Key type ._llcrypt unregistered [ 4985.024097] Key type ._llcrypt registered [ 4985.025312] Key type .llcrypt registered [ 4985.074761] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing load_modules_local [ 4985.555335] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4985.563178] alg: No test for adler32 (adler32-zlib) [ 4986.413654] Lustre: Lustre: Build Version: 2.17.0_RC1_2_g29d2772 [ 4986.501708] LNet: Added LNI 192.168.201.118@tcp [8/256/0/180] [ 4988.087189] Key type lgssc registered [ 4988.502146] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4991.836483] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4991.843953] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4993.020188] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4994.465224] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 4997.624646] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4997.763229] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4999.240450] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 5000.275518] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 5002.557098] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5002.685465] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5003.748225] LustreError: 321757:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5003.751305] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:97) [ 5003.757428] LustreError: 321757:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 5004.653578] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 5006.821258] LustreError: 321757:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5006.827213] LustreError: 321757:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 5007.588599] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5007.683465] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 5009.709996] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 5012.968375] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:65) [ 5013.779108] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 5017.398409] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 5028.322652] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5028.324027] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5028.327715] Lustre: Skipped 3 previous similar messages [ 5028.329974] Lustre: Skipped 2 previous similar messages [ 5032.193092] LustreError: 323707:0:(obd_class.h:479:obd_check_dev()) Device 10 not setup [ 5032.225913] Lustre: server umount lustre-MDT0000 complete [ 5033.440956] LustreError: 320292:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5033.446819] LustreError: 320292:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 5033.518481] LustreError: 320278:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1764749612 with bad export cookie 13240941981177616592 [ 5033.521106] LustreError: MGC192.168.201.118@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5033.522836] LustreError: 320278:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 5033.592923] LustreError: 323910:0:(obd_class.h:479:obd_check_dev()) Device 18 not setup [ 5033.595969] LustreError: 323910:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 5033.650138] Lustre: server umount lustre-MDT0001 complete [ 5045.748111] LustreError: 324110:0:(obd_class.h:479:obd_check_dev()) Device 23 not setup [ 5045.751238] LustreError: 324110:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 5045.768894] Lustre: server umount lustre-OST0000 complete [ 5057.200846] LustreError: 324312:0:(obd_class.h:479:obd_check_dev()) Device 2 not setup [ 5057.202899] LustreError: 324312:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 5057.246721] Lustre: server umount lustre-OST0001 complete [ 5062.314858] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing load_modules_local [ 5065.647699] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5065.771725] LustreError: 325489:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5065.791556] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5067.017464] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 5069.705189] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5071.077677] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 5072.003303] Lustre: 326597:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 5074.340592] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5074.470671] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5074.472759] Lustre: Skipped 1 previous similar message [ 5076.492396] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 5076.514913] LustreError: 326953:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5076.521371] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:129) [ 5079.521820] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5080.681286] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:97) [ 5081.553756] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 5084.721204] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 5085.989333] Lustre: 328444:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 5095.391875] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 5095.394282] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5095.398570] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5095.400556] Lustre: Skipped 1 previous similar message [ 5096.417262] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5096.419367] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5096.422510] Lustre: Skipped 1 previous similar message [ 5098.110601] LustreError: 328550:0:(obd_class.h:479:obd_check_dev()) Device 10 not setup [ 5098.113158] LustreError: 328550:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 5098.142663] Lustre: server umount lustre-MDT0000 complete [ 5099.466839] LustreError: 326216:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1764749678 with bad export cookie 13240941981177618286 [ 5099.472690] LustreError: MGC192.168.201.118@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5099.594590] Lustre: server umount lustre-MDT0001 complete [ 5111.733124] LustreError: 328952:0:(obd_class.h:479:obd_check_dev()) Device 23 not setup [ 5111.735878] LustreError: 328952:0:(obd_class.h:479:obd_check_dev()) Skipped 15 previous similar messages [ 5111.752862] Lustre: server umount lustre-OST0000 complete [ 5123.418632] Lustre: server umount lustre-OST0001 complete [ 5124.684858] Lustre: DEBUG MARKER: SKIP: conf-sanity test_114 skipping SLOW test 114 [ 5125.230280] Lustre: DEBUG MARKER: SKIP: conf-sanity test_115 skipping excluded test 115 [ 5125.856496] Lustre: DEBUG MARKER: == conf-sanity test 116: big size MDT support ============ 03:15:04 (1764749704) [ 5133.423410] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing load_modules_local [ 5136.597902] loop0: detected capacity change from 0 to 1099511627776 [ 5136.763769] SGI XFS with ACLs, security attributes, quota, no debug enabled [ 5136.766217] *** VALIDATE xfs *** [ 5136.770541] XFS (loop0): Mounting V5 Filesystem [ 5136.777476] XFS (loop0): Ending clean mount [ 5137.206950] loop1: detected capacity change from 0 to 18691697672192 [ 5152.706083] LDISKFS-fs (loop1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5158.794750] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_hostid [ 5162.105967] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing load_modules_local [ 5165.473735] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5167.677671] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5169.819450] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5171.979353] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5174.320825] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5174.345821] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5174.434547] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 5174.447796] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 5174.488634] Lustre: lustre-MDT0000: new disk, initializing [ 5174.515690] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5174.518832] Lustre: Skipped 1 previous similar message [ 5174.525433] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 5175.913309] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 5180.195394] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5180.219274] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5180.251874] Lustre: 333520:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 5180.278275] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 5180.281215] Lustre: Skipped 1 previous similar message [ 5180.322652] Lustre: lustre-MDT0001: new disk, initializing [ 5180.371520] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 5180.376338] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 5181.675897] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 5184.722623] Lustre: DEBUG MARKER: oleg118-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 5185.964939] Lustre: DEBUG MARKER: oleg118-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 5188.268289] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5188.294230] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5188.398994] Lustre: lustre-OST0000: new disk, initializing [ 5188.402369] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 5190.270356] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 5190.276567] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 5190.315392] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 5190.529514] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 5193.781657] Lustre: DEBUG MARKER: oleg118-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 5195.226540] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 5195.301700] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 5196.661935] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 5196.736706] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 5200.866830] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 5200.868055] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5200.868871] LustreError: Skipped 1 previous similar message [ 5200.871937] Lustre: Skipped 1 previous similar message [ 5200.873846] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5200.876482] Lustre: Skipped 4 previous similar messages [ 5203.438084] LustreError: 336464:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 5203.439893] LustreError: 336464:0:(obd_class.h:479:obd_check_dev()) Skipped 9 previous similar messages [ 5203.474874] Lustre: server umount lustre-OST0000 complete [ 5205.985027] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 5205.985092] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5205.985766] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5205.992032] Lustre: Skipped 1 previous similar message [ 5216.224188] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5216.226245] Lustre: Skipped 4 previous similar messages [ 5219.295142] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 5219.358380] Lustre: server umount lustre-MDT0000 complete [ 5220.702159] LustreError: 334301:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1764749799 with bad export cookie 13240941981177619840 [ 5220.705114] LustreError: 334301:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 6 previous similar messages [ 5220.705296] LustreError: MGC192.168.201.118@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5220.825859] Lustre: server umount lustre-MDT0001 complete [ 5223.494260] XFS (loop0): Unmounting Filesystem [ 5225.017343] Lustre: DEBUG MARKER: == conf-sanity test 117: lctl get_param return errors properly ========================================================== 03:16:43 (1764749803) [ 5227.152935] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5227.281943] LustreError: 337683:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5227.301965] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5227.303531] Lustre: Skipped 2 previous similar messages [ 5228.442968] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 5231.309366] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5232.823341] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 5234.682189] Lustre: DEBUG MARKER: oleg118-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 5235.987517] Lustre: DEBUG MARKER: oleg118-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 5238.311899] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5240.277212] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 5241.988279] Lustre: DEBUG MARKER: oleg118-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 5263.328472] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 5263.337733] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5263.338637] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5263.342755] Lustre: Skipped 1 previous similar message [ 5269.052166] LustreError: 340047:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 5269.061111] LustreError: 340047:0:(obd_class.h:479:obd_check_dev()) Skipped 17 previous similar messages [ 5269.106357] Lustre: server umount lustre-OST0000 complete [ 5272.544154] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 5285.343101] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 5285.395134] Lustre: server umount lustre-MDT0000 complete [ 5286.689204] LustreError: 338402:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1764749865 with bad export cookie 13240941981177620722 [ 5286.692789] LustreError: MGC192.168.201.118@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5286.693818] LustreError: 338402:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 5286.809185] Lustre: server umount lustre-MDT0001 complete [ 5290.718480] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing unload_modules_local [ 5291.817338] Key type lgssc unregistered [ 5291.957700] LNet: 340982:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5291.961790] LNetError: 340982:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5291.971458] LNet: Removed LNI 192.168.201.118@tcp [ 5292.315142] Key type .llcrypt unregistered [ 5292.316422] Key type ._llcrypt unregistered [ 5302.310569] Key type ._llcrypt registered [ 5302.312331] Key type .llcrypt registered [ 5302.345985] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing load_modules_local [ 5302.662987] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5302.758056] alg: No test for adler32 (adler32-zlib) [ 5303.629977] Lustre: Lustre: Build Version: 2.17.0_RC1_2_g29d2772 [ 5303.716166] LNet: Added LNI 192.168.201.118@tcp [8/256/0/180] [ 5305.295080] Key type lgssc registered [ 5305.647078] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5308.121495] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5308.128649] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5309.289723] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5310.673866] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 5313.526763] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5313.664365] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5315.137235] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 5317.085279] Lustre: DEBUG MARKER: oleg118-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 5318.268397] Lustre: DEBUG MARKER: oleg118-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 5320.575495] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5320.686707] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5322.617754] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 5324.474492] Lustre: DEBUG MARKER: oleg118-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 5339.106387] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5339.107288] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 5339.111348] Lustre: Skipped 1 previous similar message [ 5340.297614] Lustre: lustre-MDT0001: Not available for connect from 192.168.201.18@tcp (stopping) [ 5340.300971] Lustre: Skipped 1 previous similar message [ 5343.997310] LustreError: 344825:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [ 5344.018303] Lustre: server umount lustre-MDT0001 complete [ 5344.225940] LustreError: 344667:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5344.233861] LustreError: 344667:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 5345.064090] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5345.412159] LustreError: 342502:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.201.18@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5347.942595] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5347.982203] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [ 5347.984795] Lustre: Found index 1 for lustre-MDT0001, updating log [ 5348.000239] Lustre: 342495:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 5348.123733] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5349.408739] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 5353.447080] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 5357.540652] LustreError: 346022:0:(obd_mount.c:188:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [ 5357.543436] LustreError: 346022:0:(tgt_mount.c:669:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [ 5357.546649] LustreError: 342498:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [ 5357.565663] LustreError: 346026:0:(obd_config.c:1997:class_config_llog_handler()) MGC192.168.201.118@tcp: cfg command failed: rc = -17 [ 5357.571333] Lustre: cmd=cf001 0:lustre-MDT0001-osp-MDT0000 1:osp 2:lustre-MDT0000-mdtlov_UUID [ 5357.571333] [ 5652.620624] Lustre: lustre-MDT0001: Not available for connect from 192.168.201.18@tcp (stopping) [ 5653.474085] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5657.731608] Lustre: lustre-MDT0001: Not available for connect from 192.168.201.18@tcp (stopping) [ 5657.735552] Lustre: Skipped 2 previous similar messages [ 5658.557653] LustreError: 357044:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [ 5658.559726] LustreError: 357044:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 5658.586346] Lustre: server umount lustre-MDT0001 complete [ 5658.593224] LustreError: 344667:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5659.758946] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5662.611595] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5662.653550] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [ 5662.655627] Lustre: Found index 1 for lustre-MDT0001, updating log [ 5662.670286] Lustre: 342496:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 5662.809747] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5664.353490] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 5667.816970] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 5670.375045] LustreError: 358170:0:(obd_mount.c:188:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [ 5670.377551] LustreError: 358170:0:(tgt_mount.c:669:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [ 5670.381307] LustreError: 342498:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [ 5670.383105] LustreError: 342498:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [ 5670.400327] LustreError: 358173:0:(obd_config.c:689:class_setup()) Device 8 already setup (type osp) [ 5670.403778] LustreError: 358173:0:(obd_config.c:1997:class_config_llog_handler()) MGC192.168.201.118@tcp: cfg command failed: rc = -17 [ 5670.407369] Lustre: cmd=cf003 0:lustre-MDT0001-osp-MDT0000 1:lustre-MDT0001_UUID 2:192.168.201.118@tcp [ 5670.407369] [ 5969.890290] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [ 5969.891965] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5969.892923] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 5969.901607] Lustre: Skipped 2 previous similar messages [ 5973.443716] LustreError: 369358:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [ 5973.446671] LustreError: 369358:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 5973.472423] Lustre: server umount lustre-MDT0001 complete [ 5974.509686] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5975.009766] LustreError: 342505:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5975.016302] LustreError: 342505:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 5977.234597] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5977.269917] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [ 5977.272140] Lustre: Found index 1 for lustre-MDT0001, updating log [ 5977.287700] Lustre: 342496:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 5977.416180] Lustre: lustre-MDT0001: Imperative Recovery enabled, recovery window shrunk from 60-180 down to 60-180 [ 5978.864297] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 5982.692459] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 5986.277067] LustreError: 370535:0:(obd_mount.c:188:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [ 5986.280414] LustreError: 370535:0:(tgt_mount.c:669:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [ 5986.283784] LustreError: 342498:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [ 5986.286136] LustreError: 342498:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [ 5986.304612] LustreError: 370538:0:(lod_lov.c:207:lod_add_device()) lustre-MDT0001-osp-MDT0000: cannot connect to next dev lustre-MDT0001_UUID (-114) [ 5986.311321] LustreError: 370538:0:(obd_config.c:1997:class_config_llog_handler()) MGC192.168.201.118@tcp: cfg command failed: rc = -114 [ 5986.315491] Lustre: cmd=cf014 0:lustre-MDT0000-mdtlov 1:lustre-MDT0001_UUID 2:1 3:1 [ 5986.315491] [ 6295.009588] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6295.009763] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 6295.016241] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6295.021961] Lustre: Skipped 4 previous similar messages [ 6298.880597] LustreError: 381941:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup [ 6298.882644] LustreError: 381941:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 6298.916401] Lustre: server umount lustre-MDT0000 complete [ 6300.128207] LustreError: 344667:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6300.135324] LustreError: 344667:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 6300.232472] LustreError: 344125:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1764750879 with bad export cookie 10367754145814812487 [ 6300.236639] LustreError: 344125:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 6300.237256] LustreError: MGC192.168.201.118@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6305.249226] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6305.254332] Lustre: Skipped 2 previous similar messages [ 6306.366456] LustreError: 382142:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [ 6306.368713] LustreError: 382142:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 6306.422897] Lustre: server umount lustre-MDT0001 complete [ 6307.836944] Lustre: server umount lustre-OST0000 complete [ 6311.027596] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_hostid [ 6314.315726] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing load_modules_local [ 6317.926053] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6320.131384] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6322.444604] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6324.796309] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6327.300104] Lustre: DEBUG MARKER: == conf-sanity test 120: cross-target rename should not create bad symlinks ========================================================== 03:35:05 (1764750905) [ 6329.635184] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6329.657085] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6329.766413] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 6329.782915] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 6329.820771] Lustre: lustre-MDT0000: new disk, initializing [ 6329.846381] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6329.853577] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 6331.323599] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 6335.757592] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6335.782268] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 6335.820662] Lustre: 384875:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 6335.853033] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 6335.855703] Lustre: Skipped 1 previous similar message [ 6335.899249] Lustre: lustre-MDT0001: new disk, initializing [ 6335.956761] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 6335.974162] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 6335.979431] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 6337.424672] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 6340.628898] Lustre: DEBUG MARKER: oleg118-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 6341.922281] Lustre: DEBUG MARKER: oleg118-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 6344.387208] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6344.417184] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 6344.522460] Lustre: lustre-OST0000: new disk, initializing [ 6344.524675] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 6344.557712] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 6346.680706] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_default_debug -1 all [ 6349.913963] Lustre: DEBUG MARKER: oleg118-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 6353.910986] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 6353.914784] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 6353.951048] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 6359.011599] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6359.012479] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6359.016937] Lustre: Skipped 1 previous similar message [ 6359.019675] Lustre: Skipped 1 previous similar message [ 6363.777103] LustreError: 387320:0:(obd_class.h:479:obd_check_dev()) Device 22 not setup [ 6363.779720] LustreError: 387320:0:(obd_class.h:479:obd_check_dev()) Skipped 11 previous similar messages [ 6363.816572] Lustre: server umount lustre-MDT0000 complete [ 6364.129651] LustreError: 385672:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6364.136405] LustreError: 385672:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 6365.140367] LustreError: 387234:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1764750944 with bad export cookie 10367754145814814440 [ 6365.142870] LustreError: MGC192.168.201.118@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6365.144883] LustreError: 387234:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 6365.148596] LustreError: 385668:0:(osp_object.c:617:osp_attr_get()) lustre-MDT0000-osp-MDT0001: osp_attr_get update error [0x200000401:0x2:0x0]: rc = -5 [ 6365.276285] Lustre: server umount lustre-MDT0001 complete [ 6375.969144] Lustre: server umount lustre-OST0000 complete [ 6386.207000] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing set_hostid [ 6389.611053] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing load_modules_local [ 6393.425799] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6395.907137] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6398.012996] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6400.423398] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6401.053266] Lustre: DEBUG MARKER: == conf-sanity test complete, duration 6256 sec ========== 03:36:19 (1764750979) [ 6401.649535] Lustre: DEBUG MARKER: === conf-sanity: start cleanup 03:36:20 (1764750980) === [ 6402.915433] Lustre: DEBUG MARKER: === conf-sanity: finish cleanup 03:36:21 (1764750981) === [ 6417.151755] Lustre: DEBUG MARKER: oleg118-server.virtnet: executing unload_modules_local [ 6418.259238] Key type lgssc unregistered [ 6418.398360] LNet: 392823:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 6418.400516] LNetError: 392823:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 6418.408262] LNet: Removed LNI 192.168.201.118@tcp [ 6418.737283] Key type .llcrypt unregistered [ 6418.738825] Key type ._llcrypt unregistered