[ 0.000000] Linux version 4.18.0rh8.10-debug (green@maintenance) (gcc version 8.5.0 20210514 (Red Hat 8.5.0-26) (GCC)) #2 SMP Mon Jul 14 01:24:22 EDT 2025 [ 0.000000] Command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers' [ 0.000000] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256 [ 0.000000] x86/fpu: Enabled xstate features 0x7, context size is 832 bytes, using 'standard' format. [ 0.000000] signal: max sigframe size: 1776 [ 0.000000] BIOS-provided physical RAM map: [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x00000000bffcdfff] usable [ 0.000000] BIOS-e820: [mem 0x00000000bffce000-0x00000000bfffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x0000000146dfffff] usable [ 0.000000] NX (Execute Disable) protection: active [ 0.000000] SMBIOS 2.8 present. [ 0.000000] DMI: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.16.2-1.fc38 04/01/2014 [ 0.000000] Hypervisor detected: KVM [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00 [ 0.000000] kvm-clock: using sched offset of 504402644 cycles [ 0.000000] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns [ 0.000000] tsc: Detected 2399.998 MHz processor [ 0.000000] last_pfn = 0x146e00 max_arch_pfn = 0x400000000 [ 0.000000] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT [ 0.000000] last_pfn = 0xbffce max_arch_pfn = 0x400000000 [ 0.000000] found SMP MP-table at [mem 0x000f5b30-0x000f5b3f] [ 0.000000] RAMDISK: [mem 0xbcc54000-0xbffbffff] [ 0.000000] ACPI: Early table checksum verification disabled [ 0.000000] ACPI: RSDP 0x00000000000F5950 000014 (v00 BOCHS ) [ 0.000000] ACPI: RSDT 0x00000000BFFE1BB7 000034 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACP 0x00000000BFFE1A53 000074 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: DSDT 0x00000000BFFE0040 001A13 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACS 0x00000000BFFE0000 000040 [ 0.000000] ACPI: APIC 0x00000000BFFE1AC7 000090 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: HPET 0x00000000BFFE1B57 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: WAET 0x00000000BFFE1B8F 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: Reserving FACP table memory at [mem 0xbffe1a53-0xbffe1ac6] [ 0.000000] ACPI: Reserving DSDT table memory at [mem 0xbffe0040-0xbffe1a52] [ 0.000000] ACPI: Reserving FACS table memory at [mem 0xbffe0000-0xbffe003f] [ 0.000000] ACPI: Reserving APIC table memory at [mem 0xbffe1ac7-0xbffe1b56] [ 0.000000] ACPI: Reserving HPET table memory at [mem 0xbffe1b57-0xbffe1b8e] [ 0.000000] ACPI: Reserving WAET table memory at [mem 0xbffe1b8f-0xbffe1bb6] [ 0.000000] No NUMA configuration found [ 0.000000] Faking a node at [mem 0x0000000000000000-0x0000000146dfffff] [ 0.000000] NODE_DATA(0) allocated [mem 0x1465a3000-0x1465cdfff] [ 0.000000] Reserving 256MB of memory at 2752MB for crashkernel (System RAM: 4205MB) [ 0.000000] Zone ranges: [ 0.000000] DMA [mem 0x0000000000001000-0x0000000000ffffff] [ 0.000000] DMA32 [mem 0x0000000001000000-0x00000000ffffffff] [ 0.000000] Normal [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Device empty [ 0.000000] Movable zone start for each node [ 0.000000] Early memory node ranges [ 0.000000] node 0: [mem 0x0000000000001000-0x000000000009efff] [ 0.000000] node 0: [mem 0x0000000000100000-0x00000000bffcdfff] [ 0.000000] node 0: [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Zeroed struct page in unavailable ranges: 4756 pages [ 0.000000] Initmem setup node 0 [mem 0x0000000000001000-0x0000000146dfffff] [ 0.000000] ACPI: PM-Timer IO Port: 0x608 [ 0.000000] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1]) [ 0.000000] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23 [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level) [ 0.000000] Using ACPI (MADT) for SMP configuration information [ 0.000000] ACPI: HPET id: 0x8086a201 base: 0xfed00000 [ 0.000000] TSC deadline timer available [ 0.000000] smpboot: Allowing 4 CPUs, 0 hotplug CPUs [ 0.000000] kvm-guest: KVM setup pv remote TLB flush [ 0.000000] kvm-guest: setup PV sched yield [ 0.000000] PM: Registered nosave memory: [mem 0x00000000-0x00000fff] [ 0.000000] PM: Registered nosave memory: [mem 0x0009f000-0x0009ffff] [ 0.000000] PM: Registered nosave memory: [mem 0x000a0000-0x000effff] [ 0.000000] PM: Registered nosave memory: [mem 0x000f0000-0x000fffff] [ 0.000000] PM: Registered nosave memory: [mem 0xbffce000-0xbfffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xc0000000-0xfeffbfff] [ 0.000000] PM: Registered nosave memory: [mem 0xfeffc000-0xfeffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xff000000-0xfffbffff] [ 0.000000] PM: Registered nosave memory: [mem 0xfffc0000-0xffffffff] [ 0.000000] [mem 0xc0000000-0xfeffbfff] available for PCI devices [ 0.000000] Booting paravirtualized kernel on KVM [ 0.000000] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns [ 0.000000] setup_percpu: NR_CPUS:8192 nr_cpumask_bits:4 nr_cpu_ids:4 nr_node_ids:1 [ 0.000000] percpu: Embedded 63 pages/cpu s221184 r8192 d28672 u524288 [ 0.000000] kvm-guest: PV spinlocks enabled [ 0.000000] PV qspinlock hash table entries: 256 (order: 0, 4096 bytes, linear) [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 1059606 [ 0.000000] Policy zone: Normal [ 0.000000] Kernel command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] Specific versions of hardware are certified with Red Hat Enterprise Linux 8. Please see the list of hardware certified with Red Hat Enterprise Linux 8 at https://catalog.redhat.com. [ 0.000000] audit: disabled (until reboot) [ 0.000000] software IO TLB: area num 4. [ 0.000000] Memory: 2829652K/4306352K available (18435K kernel code, 11221K rwdata, 7248K rodata, 2908K init, 18040K bss, 524580K reserved, 0K cma-reserved) [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1 [ 0.000000] kmemleak: Kernel memory leak detector disabled [ 0.000000] ftrace: allocating 41240 entries in 162 pages [ 0.000000] ftrace: allocated 162 pages with 3 groups [ 0.000000] rcu: Hierarchical RCU implementation. [ 0.000000] rcu: RCU event tracing is enabled. [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=8192 to nr_cpu_ids=4. [ 0.000000] rcu: RCU callback double-/use-after-free debug enabled. [ 0.000000] Rude variant of Tasks RCU enabled. [ 0.000000] Tracing variant of Tasks RCU enabled. [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies. [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4 [ 0.000000] NR_IRQS: 524544, nr_irqs: 456, preallocated irqs: 16 [ 0.000000] random: get_random_bytes called from start_kernel+0x622/0x9a8 with crng_init=0 [ 0.001000] Console: colour *CGA 80x25 [ 0.001000] printk: console [ttyS1] enabled [ 0.001000] ACPI: Core revision 20220331 [ 0.001000] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns [ 0.001015] APIC: Switch to symmetric I/O mode setup [ 0.002316] x2apic enabled [ 0.003006] Switched APIC routing to physical x2apic. [ 0.004012] kvm-guest: setup PV IPIs [ 0.006601] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1 [ 0.007000] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 0.007022] Calibrating delay loop (skipped) preset value.. 4799.99 BogoMIPS (lpj=2399998) [ 0.008021] pid_max: default: 32768 minimum: 301 [ 0.009130] LSM: Security Framework initializing [ 0.010052] Yama: becoming mindful. [ 0.011043] SELinux: Initializing. [ 0.012065] *** VALIDATE selinux *** [ 0.020696] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, vmalloc) [ 0.024825] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, vmalloc) [ 0.025161] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.026115] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.027139] *** VALIDATE tmpfs *** [ 0.029156] *** VALIDATE proc *** [ 0.030233] *** VALIDATE cgroup *** [ 0.031006] *** VALIDATE cgroup2 *** [ 0.032253] x86/cpu: User Mode Instruction Prevention (UMIP) activated [ 0.033164] Last level iTLB entries: 4KB 0, 2MB 0, 4MB 0 [ 0.034009] Last level dTLB entries: 4KB 0, 2MB 0, 4MB 0, 1GB 0 [ 0.035028] Spectre V2 : User space: Vulnerable [ 0.036009] Speculative Store Bypass: Vulnerable [ 0.038685] debug: unmapping init [mem 0xffffffffb9a59000-0xffffffffb9a60fff] [ 0.040159] smpboot: CPU0: Intel(R) Xeon(R) CPU E5-2695 v2 @ 2.40GHz (family: 0x6, model: 0x3e, stepping: 0x4) [ 0.041731] Performance Events: IvyBridge events, full-width counters, Intel PMU driver. [ 0.042019] ... version: 2 [ 0.043010] ... bit width: 48 [ 0.044013] ... generic registers: 4 [ 0.045011] ... value mask: 0000ffffffffffff [ 0.046014] ... max period: 00007fffffffffff [ 0.047029] ... fixed-purpose events: 3 [ 0.048013] ... event mask: 000000070000000f [ 0.049306] rcu: Hierarchical SRCU implementation. [ 0.051413] smp: Bringing up secondary CPUs ... [ 0.052561] x86: Booting SMP configuration: [ 0.053025] .... node #0, CPUs: #1 #2 #3 [ 0.056422] smp: Brought up 1 node, 4 CPUs [ 0.058014] smpboot: Max logical packages: 1 [ 0.059019] smpboot: Total of 4 processors activated (19199.98 BogoMIPS) [ 0.143079] node 0 deferred pages initialised in 81ms [ 0.147097] devtmpfs: initialized [ 0.148293] x86/mm: Memory block size: 128MB [ 0.151812] gcov: version magic: 0x41383552 [ 0.154293] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns [ 0.155075] futex hash table entries: 1024 (order: 4, 65536 bytes, vmalloc) [ 0.156259] pinctrl core: initialized pinctrl subsystem [ 0.157201] [ 0.157813] ************************************************************* [ 0.158012] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.159023] ** ** [ 0.160012] ** IOMMU DebugFS SUPPORT HAS BEEN ENABLED IN THIS KERNEL ** [ 0.161017] ** ** [ 0.162015] ** This means that this kernel is built to expose internal ** [ 0.163012] ** IOMMU data structures, which may compromise security on ** [ 0.164012] ** your system. ** [ 0.165014] ** ** [ 0.166020] ** If you see this message and you are not debugging the ** [ 0.167015] ** kernel, report this immediately to your vendor! ** [ 0.168021] ** ** [ 0.169013] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.170013] ************************************************************* [ 0.172184] NET: Registered protocol family 16 [ 0.174455] DMA: preallocated 512 KiB GFP_KERNEL pool for atomic allocations [ 0.177062] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations [ 0.182077] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations [ 0.185119] cpuidle: using governor menu [ 0.187247] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5 [ 0.188000] PCI: Using configuration type 1 for base access [ 0.190142] core: PMU erratum BJ122, BV98, HSD29 worked around, HT is on [ 0.202164] HugeTLB registered 1.00 GiB page size, pre-allocated 0 pages [ 0.205054] HugeTLB registered 2.00 MiB page size, pre-allocated 0 pages [ 0.208034] cryptd: max_cpu_qlen set to 1000 [ 0.211247] ACPI: Added _OSI(Module Device) [ 0.213016] ACPI: Added _OSI(Processor Device) [ 0.218017] ACPI: Added _OSI(3.0 _SCP Extensions) [ 0.219012] ACPI: Added _OSI(Processor Aggregator Device) [ 0.223451] ACPI: 1 ACPI AML tables successfully acquired and loaded [ 0.230520] ACPI: Interpreter enabled [ 0.231054] ACPI: PM: (supports S0 S3 S4 S5) [ 0.233012] ACPI: Using IOAPIC for interrupt routing [ 0.235130] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug [ 0.238366] ACPI: Enabled 2 GPEs in block 00 to 0F [ 0.248814] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff]) [ 0.251043] acpi PNP0A03:00: _OSC: OS supports [ASPM ClockPM Segments MSI HPX-Type3] [ 0.255020] acpi PNP0A03:00: _OSC: not requesting OS control; OS requires [ExtendedConfig ASPM ClockPM MSI] [ 0.258083] acpi PNP0A03:00: fail to add MMCONFIG information, can't access extended PCI configuration space under this bridge. [ 0.263124] acpiphp: Slot [2] registered [ 0.264123] acpiphp: Slot [3] registered [ 0.265092] acpiphp: Slot [4] registered [ 0.267121] acpiphp: Slot [5] registered [ 0.269128] acpiphp: Slot [6] registered [ 0.270102] acpiphp: Slot [7] registered [ 0.271120] acpiphp: Slot [8] registered [ 0.273110] acpiphp: Slot [9] registered [ 0.274089] acpiphp: Slot [10] registered [ 0.275132] acpiphp: Slot [11] registered [ 0.277095] acpiphp: Slot [12] registered [ 0.279117] acpiphp: Slot [13] registered [ 0.281105] acpiphp: Slot [14] registered [ 0.282121] acpiphp: Slot [15] registered [ 0.284091] acpiphp: Slot [16] registered [ 0.285127] acpiphp: Slot [17] registered [ 0.287096] acpiphp: Slot [18] registered [ 0.288113] acpiphp: Slot [19] registered [ 0.290102] acpiphp: Slot [20] registered [ 0.291096] acpiphp: Slot [21] registered [ 0.292112] acpiphp: Slot [22] registered [ 0.294095] acpiphp: Slot [23] registered [ 0.295087] acpiphp: Slot [24] registered [ 0.296090] acpiphp: Slot [25] registered [ 0.297071] acpiphp: Slot [26] registered [ 0.298091] acpiphp: Slot [27] registered [ 0.299108] acpiphp: Slot [28] registered [ 0.301092] acpiphp: Slot [29] registered [ 0.302104] acpiphp: Slot [30] registered [ 0.303088] acpiphp: Slot [31] registered [ 0.305066] PCI host bridge to bus 0000:00 [ 0.306019] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window] [ 0.309035] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window] [ 0.311027] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window] [ 0.314032] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window] [ 0.317026] pci_bus 0000:00: root bus resource [mem 0x180000000-0x1ffffffff window] [ 0.319023] pci_bus 0000:00: root bus resource [bus 00-ff] [ 0.321174] pci 0000:00:00.0: [8086:1237] type 00 class 0x060000 [ 0.325437] pci 0000:00:01.0: [8086:7000] type 00 class 0x060100 [ 0.331979] pci 0000:00:01.1: [8086:7010] type 00 class 0x010180 [ 0.341752] pci 0000:00:01.1: reg 0x20: [io 0xc320-0xc32f] [ 0.348011] pci 0000:00:01.1: legacy IDE quirk: reg 0x10: [io 0x01f0-0x01f7] [ 0.350019] pci 0000:00:01.1: legacy IDE quirk: reg 0x14: [io 0x03f6] [ 0.352036] pci 0000:00:01.1: legacy IDE quirk: reg 0x18: [io 0x0170-0x0177] [ 0.355018] pci 0000:00:01.1: legacy IDE quirk: reg 0x1c: [io 0x0376] [ 0.357492] pci 0000:00:01.3: [8086:7113] type 00 class 0x068000 [ 0.360766] pci 0000:00:01.3: quirk: [io 0x0600-0x063f] claimed by PIIX4 ACPI [ 0.363038] pci 0000:00:01.3: quirk: [io 0x0700-0x070f] claimed by PIIX4 SMB [ 0.365622] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 [ 0.370014] pci 0000:00:02.0: reg 0x10: [io 0xc300-0xc31f] [ 0.383017] pci 0000:00:02.0: reg 0x20: [mem 0xfebe4000-0xfebe7fff 64bit pref] [ 0.388025] pci 0000:00:02.0: reg 0x30: [mem 0xfeb80000-0xfebbffff pref] [ 0.396124] pci 0000:00:05.0: [1af4:1001] type 00 class 0x010000 [ 0.403019] pci 0000:00:05.0: reg 0x10: [io 0xc000-0xc07f] [ 0.408016] pci 0000:00:05.0: reg 0x14: [mem 0xfebc0000-0xfebc0fff] [ 0.426017] pci 0000:00:05.0: reg 0x20: [mem 0xfebe8000-0xfebebfff 64bit pref] [ 0.440268] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 [ 0.446016] pci 0000:00:06.0: reg 0x10: [io 0xc080-0xc0ff] [ 0.453017] pci 0000:00:06.0: reg 0x14: [mem 0xfebc1000-0xfebc1fff] [ 0.473016] pci 0000:00:06.0: reg 0x20: [mem 0xfebec000-0xfebeffff 64bit pref] [ 0.484721] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 [ 0.491030] pci 0000:00:07.0: reg 0x10: [io 0xc100-0xc17f] [ 0.500019] pci 0000:00:07.0: reg 0x14: [mem 0xfebc2000-0xfebc2fff] [ 0.513022] pci 0000:00:07.0: reg 0x20: [mem 0xfebf0000-0xfebf3fff 64bit pref] [ 0.522397] pci 0000:00:08.0: [1af4:1001] type 00 class 0x010000 [ 0.530017] pci 0000:00:08.0: reg 0x10: [io 0xc180-0xc1ff] [ 0.542017] pci 0000:00:08.0: reg 0x14: [mem 0xfebc3000-0xfebc3fff] [ 0.561018] pci 0000:00:08.0: reg 0x20: [mem 0xfebf4000-0xfebf7fff 64bit pref] [ 0.570000] pci 0000:00:09.0: [1af4:1001] type 00 class 0x010000 [ 0.579015] pci 0000:00:09.0: reg 0x10: [io 0xc200-0xc27f] [ 0.585016] pci 0000:00:09.0: reg 0x14: [mem 0xfebc4000-0xfebc4fff] [ 0.604016] pci 0000:00:09.0: reg 0x20: [mem 0xfebf8000-0xfebfbfff 64bit pref] [ 0.613354] pci 0000:00:0a.0: [1af4:1001] type 00 class 0x010000 [ 0.625026] pci 0000:00:0a.0: reg 0x10: [io 0xc280-0xc2ff] [ 0.630021] pci 0000:00:0a.0: reg 0x14: [mem 0xfebc5000-0xfebc5fff] [ 0.642042] pci 0000:00:0a.0: reg 0x20: [mem 0xfebfc000-0xfebfffff 64bit pref] [ 0.657218] ACPI: PCI: Interrupt link LNKA configured for IRQ 10 [ 0.660375] ACPI: PCI: Interrupt link LNKB configured for IRQ 10 [ 0.662320] ACPI: PCI: Interrupt link LNKC configured for IRQ 11 [ 0.665356] ACPI: PCI: Interrupt link LNKD configured for IRQ 11 [ 0.667250] ACPI: PCI: Interrupt link LNKS configured for IRQ 9 [ 0.671177] iommu: Default domain type: Passthrough [ 0.672000] SCSI subsystem initialized [ 0.672000] ACPI: bus type USB registered [ 0.672160] usbcore: registered new interface driver usbfs [ 0.673000] usbcore: registered new interface driver hub [ 0.676086] usbcore: registered new device driver usb [ 0.677190] pps_core: LinuxPPS API ver. 1 registered [ 0.679010] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti [ 0.682114] PTP clock support registered [ 0.683131] EDAC MC: Ver: 3.0.0 [ 0.684160] PCI: Using ACPI for IRQ routing [ 0.687028] NetLabel: Initializing [ 0.688009] NetLabel: domain hash size = 128 [ 0.690011] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO [ 0.692100] NetLabel: unlabeled traffic allowed by default [ 0.694177] vgaarb: loaded [ 0.696305] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0 [ 0.698017] hpet0: 3 comparators, 64-bit 100.000000 MHz counter [ 0.710793] clocksource: Switched to clocksource kvm-clock [ 0.820895] VFS: Disk quotas dquot_6.6.0 [ 0.822580] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) [ 0.825297] *** VALIDATE ramfs *** [ 0.826706] *** VALIDATE hugetlbfs *** [ 0.828349] pnp: PnP ACPI init [ 0.830774] pnp: PnP ACPI: found 6 devices [ 0.849060] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns [ 0.852196] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window] [ 0.855633] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window] [ 0.857147] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window] [ 0.858781] pci_bus 0000:00: resource 7 [mem 0xc0000000-0xfebfffff window] [ 0.860393] pci_bus 0000:00: resource 8 [mem 0x180000000-0x1ffffffff window] [ 0.863096] NET: Registered protocol family 2 [ 0.865601] IP idents hash table entries: 131072 (order: 8, 1048576 bytes, vmalloc) [ 0.870589] tcp_listen_portaddr_hash hash table entries: 4096 (order: 5, 163840 bytes, vmalloc) [ 0.874503] TCP established hash table entries: 65536 (order: 7, 524288 bytes, vmalloc) [ 0.879636] TCP bind hash table entries: 65536 (order: 9, 2097152 bytes, vmalloc) [ 0.882856] TCP: Hash tables configured (established 65536 bind 65536) [ 0.886081] MPTCP token hash table entries: 8192 (order: 6, 393216 bytes, vmalloc) [ 0.890364] UDP hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 0.894087] UDP-Lite hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 0.897572] NET: Registered protocol family 1 [ 0.906568] RPC: Registered named UNIX socket transport module. [ 0.909311] RPC: Registered udp transport module. [ 0.911560] RPC: Registered tcp transport module. [ 0.913537] RPC: Registered tcp NFSv4.1 backchannel transport module. [ 0.916217] NET: Registered protocol family 44 [ 0.918067] pci 0000:00:00.0: Limiting direct PCI/PCI transfers [ 0.919961] pci 0000:00:01.0: PIIX3: Enabling Passive Release [ 0.922050] pci 0000:00:01.0: Activating ISA DMA hang workarounds [ 0.924385] PCI: CLS 0 bytes, default 64 [ 0.926735] Unpacking initramfs... [ 2.614170] debug: unmapping init [mem 0xffff89e87cc54000-0xffff89e87ffbffff] [ 2.620680] PCI-DMA: Using software bounce buffering for IO (SWIOTLB) [ 2.623120] software IO TLB: mapped [mem 0x00000000a8000000-0x00000000ac000000] (64MB) [ 2.627151] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 4.003872] Initialise system trusted keyrings [ 4.005939] Key type blacklist registered [ 4.008825] workingset: timestamp_bits=36 max_order=20 bucket_order=0 [ 4.019232] zbud: loaded [ 4.023584] *** VALIDATE nfs *** [ 4.024976] *** VALIDATE nfs4 *** [ 4.026566] pstore: using deflate compression [ 4.032612] Platform Keyring initialized [ 4.284745] NET: Registered protocol family 38 [ 4.292789] Key type asymmetric registered [ 4.294739] Asymmetric key parser 'x509' registered [ 4.296996] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 247) [ 4.300517] io scheduler mq-deadline registered [ 4.301849] io scheduler kyber registered [ 4.304892] io scheduler bfq registered [ 4.306992] atomic64_test: passed for x86-64 platform with CX8 and with SSE [ 4.311973] shpchp: Standard Hot Plug PCI Controller Driver version: 0.4 [ 4.317109] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0 [ 4.320126] ACPI: Power Button [PWRF] [ 4.511062] ACPI: \_SB_.LNKB: Enabled at IRQ 10 [ 4.685524] ACPI: \_SB_.LNKA: Enabled at IRQ 11 [ 5.066143] ACPI: \_SB_.LNKC: Enabled at IRQ 11 [ 5.306986] ACPI: \_SB_.LNKD: Enabled at IRQ 10 [ 5.639449] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled [ 5.707704] 00:03: ttyS1 at I/O 0x2f8 (irq = 3, base_baud = 115200) is a 16550A [ 5.765592] 00:04: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A [ 5.785118] Non-volatile memory driver v1.3 [ 5.786942] Linux agpgart interface v0.103 [ 5.927566] virtio_blk virtio1: [vda] 133848 512-byte logical blocks (68.5 MB/65.4 MiB) [ 5.943495] vda: detected capacity change from 0 to 68530176 [ 5.980631] virtio_blk virtio2: [vdb] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) [ 5.983809] vdb: detected capacity change from 0 to 1073741824 [ 6.035547] virtio_blk virtio3: [vdc] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 6.049126] vdc: detected capacity change from 0 to 2621440000 [ 6.093783] virtio_blk virtio4: [vdd] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 6.105569] vdd: detected capacity change from 0 to 2621440000 [ 6.161645] virtio_blk virtio5: [vde] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 6.174399] vde: detected capacity change from 0 to 4294967296 [ 6.211510] virtio_blk virtio6: [vdf] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 6.214158] vdf: detected capacity change from 0 to 4294967296 [ 6.227787] libphy: Fixed MDIO Bus: probed [ 6.250230] usbcore: registered new interface driver usbserial_generic [ 6.265243] usbserial: USB Serial support registered for generic [ 6.268347] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12 [ 6.282561] serio: i8042 KBD port at 0x60,0x64 irq 1 [ 6.286560] serio: i8042 AUX port at 0x60,0x64 irq 12 [ 6.299334] mousedev: PS/2 mouse device common for all mice [ 6.304088] rtc_cmos 00:05: RTC can wake from S4 [ 6.316452] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input1 [ 6.319130] rtc_cmos 00:05: registered as rtc0 [ 6.353879] rtc_cmos 00:05: alarms up to one day, y3k, 242 bytes nvram, hpet irqs [ 6.356754] intel_pstate: CPU model not supported [ 6.369301] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input4 [ 6.376396] hid: raw HID events driver (C) Jiri Kosina [ 6.401074] usbcore: registered new interface driver usbhid [ 6.403175] usbhid: USB HID core driver [ 6.404750] drop_monitor: Initializing network drop monitor service [ 6.407614] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input3 [ 6.410972] Initializing XFRM netlink socket [ 6.417443] NET: Registered protocol family 10 [ 6.430949] Segment Routing with IPv6 [ 6.432585] NET: Registered protocol family 17 [ 6.436197] mpls_gso: MPLS GSO support [ 6.445356] RAS: Correctable Errors collector initialized. [ 6.448237] AVX version of gcm_enc/dec engaged. [ 6.449709] AES CTR mode by8 optimization enabled [ 6.676410] sched_clock: Marking stable (6676334078, 0)->(7616541073, -940206995) [ 6.679980] registered taskstats version 1 [ 6.684346] Loading compiled-in X.509 certificates [ 6.686126] zswap: loaded using pool lzo/zbud [ 6.758875] Key type big_key registered [ 6.835705] Key type encrypted registered [ 6.838815] ima: No TPM chip found, activating TPM-bypass! [ 6.842826] ima: Allocated hash algorithm: sha1 [ 6.847171] ima: No architecture policies found [ 6.853284] evm: Initialising EVM extended attributes: [ 6.858173] evm: security.selinux [ 6.860460] evm: security.ima [ 6.864403] evm: security.capability [ 6.867866] evm: HMAC attrs: 0x1 [ 6.876702] rtc_cmos 00:05: setting system clock to 2025-11-16 23:05:21 UTC (1763334321) [ 6.886388] debug: unmapping init [mem 0xffffffffbaa03000-0xffffffffbabfffff] [ 6.897969] debug: unmapping init [mem 0xffffffffb9782000-0xffffffffb9a58fff] [ 6.913943] Write protecting the kernel read-only data: 28672k [ 6.922672] debug: unmapping init [mem 0xffffffffb7e03000-0xffffffffb7ffffff] [ 6.933143] debug: unmapping init [mem 0xffffffffb8714000-0xffffffffb87fffff] [ 7.017989] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 7.028433] systemd[1]: Detected virtualization kvm. [ 7.030265] systemd[1]: Detected architecture x86-64. [ 7.032196] systemd[1]: Running in initial RAM disk. Welcome to Rocky Linux 8.10 (Green Obsidian) dracut-049-233.git20240115.el8 (Initramfs)! [ 7.072770] systemd[1]: No hostname configured. [ 7.075138] systemd[1]: Set hostname to . [ 7.079506] random: systemd: uninitialized urandom read (16 bytes read) [ 7.094064] systemd[1]: Initializing machine ID from random generator. [ 7.255123] random: ln: uninitialized urandom read (6 bytes read) [ 7.420272] random: systemd: uninitialized urandom read (16 bytes read) [ 7.423575] systemd[1]: Listening on Journal Socket. [ OK ] Listening on Journal Socket. [ 7.437479] systemd[1]: Starting Create list of required static device nodes for the current kernel... Starting Create list of required st…ce nodes for the current kernel... [ 7.456319] systemd[1]: Reached target Timers. [ OK ] Reached target Timers. [ OK ] Started Memstrack Anylazing Service. [ OK ] Reached target Swap. [ OK ] Reached target Initrd Root Device. Starting Setup Virtual Console... [ OK ] Listening on Journal Socket (/dev/log). Starting Journal Service... [ OK ] Listening on udev Control Socket. [ OK ] Reached target Slices. [ OK ] Listening on udev Kernel Socket. [ OK ] Reached target Sockets. Starting Apply Kernel Variables... [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Reached target Local Encrypted Volumes. [ OK ] Reached target Paths. [ OK ] Reached target Local File Systems. Starting Create Volatile Files and Directories... [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Started Setup Virtual Console. [ OK ] Started Apply Kernel Variables. [ OK ] Started Create Volatile Files and Directories. Starting dracut cmdline hook... Starting Create Static Device Nodes in /dev... [ OK ] Started Create Static Device Nodes in /dev. [ OK ] Started Journal Service. [ OK ] Started dracut cmdline hook. Starting dracut pre-udev hook... [ 8.814520] device-mapper: uevent: version 1.0.3 [ 8.823607] device-mapper: ioctl: 4.46.0-ioctl (2022-02-22) initialised: dm-devel@redhat.com [ OK ] Started dracut pre-udev hook. Starting udev Kernel Device Manager... [ OK ] Started udev Kernel Device Manager. Starting dracut pre-trigger hook... [ OK ] Started dracut pre-trigger hook. Starting udev Coldplug all Devices... Mounting Kernel Configuration File System... [ OK ] Mounted Kernel Configuration File System. [ OK ] Started udev Coldplug all Devices. [ OK ] Reached target System Initialization. [ OK ] Reached target Basic System. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. Starting dracut initqueue hook... [ 10.349570] virtio_net virtio0 ens2: renamed from eth0 [ 10.376574] random: fast init done [ 10.716987] scsi host0: ata_piix [ 10.805505] scsi host1: ata_piix [ 10.811067] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc320 irq 14 [ 10.815768] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc328 irq 15 [ 15.379812] random: crng init done [ 15.381488] random: 7 urandom warning(s) missed due to ratelimiting [ 15.848231] dracut-initqueue[586]: RTNETLINK answers: File exists Starting nbd nbd0... [ OK ] Started nbd nbd0. [ OK ] Started dracut initqueue hook. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Mounting /sysroot... [ 17.626517] EXT4-fs (nbd0): mounted filesystem with ordered data mode. Opts: (null) [ OK ] Mounted /sysroot. [ OK ] Reached target Initrd Root File System. Starting Reload Configuration from the Real Root... [ OK ] Started Reload Configuration from the Real Root. [ OK ] Reached target Initrd File Systems. [ OK ] Reached target Initrd Default Target. Starting dracut pre-pivot and cleanup hook... [ OK ] Started dracut pre-pivot and cleanup hook. Starting Cleaning Up and Shutting Down Daemons... [ OK ] Stopped target Timers. Stopping Hardware RNG Entropy Gatherer Daemon... [ OK ] Stopped dracut pre-pivot and cleanup hook. [ OK ] Stopped target Initrd Default Target. [ OK ] Stopped target Initrd Root Device. [ OK ] Stopped target Remote File Systems. [ OK ] Stopped target Remote File Systems (Pre). [ OK ] Stopped dracut initqueue hook. [ OK ] Stopped Hardware RNG Entropy Gatherer Daemon. [ OK ] Stopped target Basic System. [ OK ] Stopped target Sockets. [ OK ] Stopped target Paths. [ OK ] Stopped target Slices. [ OK ] Stopped target System Initialization. [ OK ] Stopped Apply Kernel Variables. [ OK ] Stopped target Swap. [ OK ] Stopped Create Volatile Files and Directories. [ OK ] Stopped target Local File Systems. [ OK ] Stopped target Local Encrypted Volumes. [ OK ] Stopped Dispatch Password Requests to Console Directory Watch. [ OK ] Stopped udev Coldplug all Devices. [ OK ] Stopped dracut pre-trigger hook. Stopping udev Kernel Device Manager... [ OK ] Started Cleaning Up and Shutting Down Daemons. [ OK ] Stopped udev Kernel Device Manager. [ OK ] Stopped dracut pre-udev hook. [ OK ] Stopped dracut cmdline hook. [ OK ] Stopped Create Static Device Nodes in /dev. [ OK ] Stopped Create list of required sta…vice nodes for the current kernel. [ OK ] Closed udev Control Socket. [ OK ] Closed udev Kernel Socket. Starting Cleanup udevd DB... [ OK ] Started Cleanup udevd DB. [ OK ] Reached target Switch Root. Starting Switch Root... [ 19.619104] printk: systemd: 26 output lines suppressed due to ratelimiting [ 20.342857] SELinux: Disabled at runtime. [ 20.434983] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 20.445269] systemd[1]: Detected virtualization kvm. [ 20.447495] systemd[1]: Detected architecture x86-64. Welcome to Rocky Linux 8.10 (Green Obsidian)! [ 21.437720] systemd[1]: initrd-switch-root.service: Succeeded. [ 21.441295] systemd[1]: Stopped Switch Root. [ OK ] Stopped Switch Root. [ 21.457830] systemd[1]: systemd-journald.service: Service has no hold-off time (RestartSec=0), scheduling restart. [ 21.465764] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. [ 21.473049] systemd[1]: Stopped Journal Service. [ OK ] Stopped Journal Service. [ 21.481498] systemd[1]: Starting Journal Service... Starting Journal Service... [ 21.493101] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. [ OK ] Started Dispatch Password Requests to Console Directory Watch. Mounting POSIX Message Queue File System... [ OK ] Created slice User and Session Slice. [FAILED] Failed to set up automount Arbitrar…rmats File System Automount Point. See 'systemctl status proc-sys-fs-binfmt_misc.automount' for details. Activating swap /dev/disk/by-label/SWAP... Mounting Kernel Debug File System... Starting Remount Root and Kernel File Systems... [ OK ] Listening on RPCbind Server Activation Socket.[ 21.601352] Adding 1048572k swap on /dev/vdb. Priority:-2 extents:1 across:1048572k FS [ OK ] Created slice system-getty.slice. Starting Create list of required st…ce nodes for the current kernel... Mounting Huge Pages File System... [ OK ] Started Forward Password Requests to Wall Directory Watch. [ OK ] Reached target Paths. [ OK ] Reached target Local Encrypted Volumes. [ OK ] Reached target rpc_pipefs.target. Starting Apply Kernel Variables... [ OK ] Listening on udev Control Socket. [ OK ] Created slice system-serial\x2dgetty.slice. [ OK ] Listening on udev Kernel Socket. Starting udev Coldplug all Devices... [ OK ] Reached target RPC Port Mapper. [ OK ] Created slice system-sshd\x2dkeygen.slice. [ OK ] Listening on initctl Compatibility Named Pipe. [ OK ] Stopped target Switch Root. [ OK ] Stopped target Initrd File Systems. [ OK ] Stopped target Initrd Root File System. [ OK ] Reached target Slices. [ OK ] Listening on Process Core Dump Socket. [ OK ] Started Journal Service. [ OK ] Mounted POSIX Message Queue File System. [ OK ] Activated swap /dev/disk/by-label/SWAP. [ OK ] Mounted Kernel Debug File System. [FAILED] Failed to start Remount Root and Kernel File Systems. See 'systemctl status systemd-remount-fs.service' for details. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Mounted Huge Pages File System. [ OK ] Started Apply Kernel Variables. Starting Create Static Device Nodes in /dev... Starting Configure read-only root support... [ OK ] Reached target Swap. Starting Flush Journal to Persistent Storage... [ OK ] Started Flush Journal to Persistent Storage. [ OK ] Started Create Static Device Nodes in /dev. Starting udev Kernel Device Manager... [ OK ] Reached target Local File Systems (Pre). Mounting /mnt... Mounting /home/green/git/lustre-release... [ OK ] Started udev Coldplug all Devices. [ OK ] Mounted /mnt. [ 22.296924] squashfs: version 4.0 (2009/01/31) Phillip Lougher [ OK ] Mounted /home/green/git/lustre-release. [ OK ] Started udev Kernel Device Manager. [ 22.999241] input: PC Speaker as /devices/platform/pcspkr/input/input5 [ 23.134851] piix4_smbus 0000:00:01.3: SMBus Host Controller at 0x700, revision 0 [ 23.561690] RAPL PMU: API unit is 2^-32 Joules, 0 fixed counters, 10737418240 ms ovfl timer [ 23.603294] EDAC sbridge: Ver: 1.1.2 [ 27.603926] Key type dns_resolver registered [* ] A start job is running for Configur…-only root support (6s / no limit)[ 28.069974] NFS: Registering the id_resolver key type [ 28.073479] Key type id_resolver registered [ 28.075140] Key type id_legacy registered [** ] A start job is running for Configur…-only root support (7s / no limit) [ OK ] Started Configure read-only root support. Starting Load/Save Random Seed... [ OK ] Reached target Local File Systems. Starting Mark the need to relabel after reboot... Starting Rebuild Dynamic Linker Cache... Starting Create Volatile Files and Directories... [ OK ] Started Load/Save Random Seed. [ OK ] Started Mark the need to relabel after reboot. [ OK ] Started Create Volatile Files and Directories. Starting RPC Bind... Starting Update UTMP about System Boot/Shutdown... [ OK ] Started Update UTMP about System Boot/Shutdown. [ OK ] Started RPC Bind. [ OK ] Started Rebuild Dynamic Linker Cache. Starting Update is Completed... [ OK ] Started Update is Completed. [ OK ] Reached target System Initialization. [ OK ] Started dnf makecache --timer. [ OK ] Started Daily Cleanup of Temporary Directories. [ OK ] Listening on D-Bus System Message Bus Socket. [ OK ] Reached target Sockets. [ OK ] Reached target Basic System. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. [ OK ] Started irqbalance daemon. Starting Login Service... Starting Restore /run/initramfs on shutdown... [ OK ] Started daily update of the root trust anchor for DNSSEC. [ OK ] Reached target Timers. [ OK ] Started D-Bus System Message Bus. [ OK ] Reached target sshd-keygen.target. Starting Network Manager... [ OK ] Started Restore /run/initramfs on shutdown. [ OK ] Started Login Service. [ OK ] Started Network Manager. [ OK ] Reached target Network. Starting OpenSSH server daemon... Starting GSSAPI Proxy Daemon... Starting Dynamic System Tuning Daemon... Starting Network Manager Wait Online... [ OK ] Started OpenSSH server daemon. Starting Hostname Service... [ OK ] Started GSSAPI Proxy Daemon. [ OK ] Reached target NFS client services. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Starting Permit User Sessions... [ OK ] Started Permit User Sessions. [ OK ] Started Serial Getty on ttyS1. [ OK ] Started Getty on tty1. [ OK ] Started Command Scheduler. [ OK ] Started Serial Getty on ttyS0. [ OK ] Reached target Login Prompts. [ OK ] Started Hostname Service. Starting Network Manager Script Dispatcher Service... [ OK ] Started Network Manager Script Dispatcher Service. [ OK ] Started Network Manager Wait Online. [ OK ] Reached target Network is Online. Starting System Logging Service... Starting Notify NFS peers of a restart... Starting Crash recovery kernel arming... [ OK ] Started Notify NFS peers of a restart. [ OK ] Started System Logging Service. Starting Authorization Manager... Rocky Linux 8.10 (Green Obsidian) Kernel 4.18.0rh8.10-debug on an x86_64 oleg217-server login: [ 57.904142] hrtimer: interrupt took 2285384 ns [ 73.637491] libcfs: loading out-of-tree module taints kernel. [ 73.659343] Key type ._llcrypt registered [ 73.661497] Key type .llcrypt registered [ 73.771794] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing set_hostid [ 89.624255] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing load_modules_local [ 90.669564] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 90.677728] alg: No test for adler32 (adler32-zlib) [ 92.106802] Lustre: Lustre: Build Version: 2.16.61_54_g3c0e0e6 [ 92.985701] LNet: Added LNI 192.168.202.117@tcp [8/256/0/180] [ 94.689701] Key type lgssc registered [ 96.487911] Lustre: Echo OBD driver; http://www.lustre.org/ [ 108.785759] ZFS: Loaded module v2.3.2-1, ZFS pool version 5000, ZFS filesystem version 5 [ 109.435682] LDISKFS-fs (vdc): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 115.882751] LDISKFS-fs (vdd): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 121.543087] LDISKFS-fs (vde): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 128.041644] LDISKFS-fs (vdf): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 140.081722] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing load_modules_local [ 147.851305] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 147.899310] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 147.913070] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 149.064919] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 149.088090] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 149.136399] Lustre: lustre-MDT0000: new disk, initializing [ 149.188501] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 149.198901] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 151.371500] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 159.886372] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 159.936636] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 159.981088] Lustre: 6520:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 160.001226] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 160.003579] Lustre: Skipped 1 previous similar message [ 160.057212] Lustre: lustre-MDT0001: new disk, initializing [ 160.114780] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 160.146612] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 160.153340] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 162.224066] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 165.747421] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 173.826182] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 173.880472] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 174.064494] Lustre: lustre-OST0000: new disk, initializing [ 174.067126] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 174.147567] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 178.706746] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 178.711654] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 178.774222] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 179.237328] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 191.659491] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 191.755673] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 191.859456] Lustre: lustre-OST0001: new disk, initializing [ 191.872907] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 191.939835] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 196.775358] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 199.738744] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 199.761419] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 199.868351] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 208.471325] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 221.076737] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 231.107177] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing check_logdir /tmp/testlogs/ [ 234.806513] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing yml_node [ 239.133094] Lustre: DEBUG MARKER: Client: 2.16.61.54 [ 241.701833] Lustre: DEBUG MARKER: MDS: 2.16.61.54 [ 244.409657] Lustre: DEBUG MARKER: OSS: 2.16.61.54 [ 245.881174] Lustre: DEBUG MARKER: -----============= acceptance-small: conf-sanity ============----- Sun Nov 16 18:09:19 EST 2025 [ 259.733682] Lustre: DEBUG MARKER: excepting tests: 102 106 115 32newtarball [ 261.132411] Lustre: DEBUG MARKER: skipping tests SLOW=no: 45 69 106 111 114 [ 276.448828] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 276.450244] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 276.457215] Lustre: Skipped 2 previous similar messages [ 276.459918] Lustre: Skipped 1 previous similar message [ 281.575452] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 281.583215] Lustre: Skipped 5 previous similar messages [ 281.731474] LustreError: 12223:0:(obd_class.h:479:obd_check_dev()) Device 28 not setup [ 281.821944] Lustre: server umount lustre-MDT0000 complete [ 284.813034] LustreError: 6511:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763334599 with bad export cookie 5223382813477262476 [ 284.816868] LustreError: MGC192.168.202.117@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 284.829959] LustreError: 6511:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 286.689851] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 286.695502] Lustre: Skipped 1 previous similar message [ 286.701994] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 286.714123] Lustre: Skipped 1 previous similar message [ 289.760791] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 289.763859] Lustre: Skipped 1 previous similar message [ 291.070211] LustreError: 12425:0:(obd_class.h:479:obd_check_dev()) Device 27 not setup [ 291.086254] LustreError: 12425:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 291.218720] Lustre: server umount lustre-MDT0001 complete [ 294.312092] LustreError: 12626:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 294.321353] LustreError: 12626:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 294.357912] Lustre: server umount lustre-OST0000 complete [ 297.431099] LustreError: 12827:0:(obd_class.h:479:obd_check_dev()) Device 2 not setup [ 297.433774] LustreError: 12827:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 297.557738] Lustre: server umount lustre-OST0001 complete [ 302.457535] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing set_hostid [ 310.421799] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing load_modules_local [ 318.839875] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 323.885314] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 329.321127] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 334.192910] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 339.961674] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 340.075312] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 340.336993] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 340.395986] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 340.515316] Lustre: lustre-MDT0000: new disk, initializing [ 340.626464] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 340.648469] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 344.514931] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing set_default_debug -1 all [ 354.105538] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 354.163762] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 354.223444] Lustre: 15013:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 354.309402] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 354.312383] Lustre: Skipped 1 previous similar message [ 354.458826] Lustre: lustre-MDT0001: new disk, initializing [ 354.617354] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 354.688512] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 354.706640] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 358.950597] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing set_default_debug -1 all [ 365.497298] Lustre: DEBUG MARKER: oleg217-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 368.966345] Lustre: DEBUG MARKER: oleg217-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 374.778186] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 374.828438] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 375.019875] Lustre: lustre-OST0000: new disk, initializing [ 375.026399] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 375.088561] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 376.444526] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 376.455622] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 376.527243] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 378.775662] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing set_default_debug -1 all [ 384.101772] Lustre: DEBUG MARKER: oleg217-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 387.518559] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 387.742566] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 390.810885] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 390.952844] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 397.284135] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 397.287680] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 397.288676] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 397.304439] Lustre: Skipped 1 previous similar message [ 397.317248] LustreError: Skipped 1 previous similar message [ 398.466113] LustreError: 17951:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 398.480066] LustreError: 17951:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 398.558552] Lustre: server umount lustre-OST0000 complete [ 402.403531] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 402.414688] Lustre: Skipped 1 previous similar message [ 407.522271] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 407.525337] Lustre: Skipped 4 previous similar messages [ 415.203201] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 415.305295] LustreError: 18152:0:(obd_class.h:479:obd_check_dev()) Device 22 not setup [ 415.312632] LustreError: 18152:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 415.383294] Lustre: server umount lustre-MDT0000 complete [ 417.770475] LustreError: 15026:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 417.801114] LustreError: 15026:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 6 previous similar messages [ 418.595870] LustreError: 15795:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763334733 with bad export cookie 5223382813477264009 [ 418.596364] LustreError: MGC192.168.202.117@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 418.613196] LustreError: 15795:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 418.923274] Lustre: server umount lustre-MDT0001 complete [ 422.760356] Lustre: DEBUG MARKER: == conf-sanity test 76a: set permanent params with lctl across mounts ========================================================== 18:12:16 (1763334736) [ 427.067116] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 427.349733] LustreError: 18881:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 427.399168] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 429.577391] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing set_default_debug -1 all [ 432.617797] LustreError: 18881:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 435.416119] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 438.832727] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing set_default_debug -1 all [ 442.787911] Lustre: DEBUG MARKER: oleg217-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 445.493240] Lustre: DEBUG MARKER: oleg217-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 450.465327] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 454.109155] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing set_default_debug -1 all [ 457.391187] Lustre: DEBUG MARKER: oleg217-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 463.531772] Lustre: Modifying parameter general.osc.*.max_dirty_mb=452 in log params [ 469.647361] LustreError: 21181:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup [ 469.650303] LustreError: 21181:0:(obd_class.h:479:obd_check_dev()) Skipped 13 previous similar messages [ 469.703277] Lustre: server umount lustre-MDT0000 complete [ 471.523573] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 471.523605] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 471.527346] LustreError: 18876:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 471.541026] Lustre: Skipped 2 previous similar messages [ 471.780311] LustreError: 18860:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763334786 with bad export cookie 5223382813477264891 [ 471.783737] LustreError: MGC192.168.202.117@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 471.785484] LustreError: 18860:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 474.068457] Lustre: server umount lustre-MDT0001 complete [ 486.971992] Lustre: server umount lustre-OST0000 complete [ 501.092908] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing load_modules_local [ 508.550521] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 508.899178] LustreError: 22911:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 508.925489] LustreError: 22911:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 509.000611] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 509.006123] Lustre: Skipped 2 previous similar messages [ 512.477826] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing set_default_debug -1 all [ 519.147572] LustreError: 22910:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 519.162748] LustreError: 22910:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 519.957524] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 523.259345] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing set_default_debug -1 all [ 525.550505] Lustre: Setting parameter general.debug_raw_pointers=Y in log params [ 531.023390] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 535.008299] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing set_default_debug -1 all [ 541.159595] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 541.219826] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 541.314963] Lustre: lustre-OST0001: new disk, initializing [ 541.319691] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 542.813313] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 542.821356] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 542.948327] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 546.025462] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing set_default_debug -1 all [ 553.915544] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 556.977405] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 571.881654] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 571.882454] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 571.882512] Lustre: Skipped 2 previous similar messages [ 571.909807] Lustre: Skipped 1 previous similar message [ 576.485394] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 577.736880] LustreError: 26390:0:(obd_class.h:479:obd_check_dev()) Device 28 not setup [ 577.741689] LustreError: 26390:0:(obd_class.h:479:obd_check_dev()) Skipped 17 previous similar messages [ 577.846226] Lustre: server umount lustre-MDT0000 complete [ 580.928599] LustreError: 22891:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763334895 with bad export cookie 5223382813477265731 [ 580.929127] LustreError: MGC192.168.202.117@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 580.935771] LustreError: 22891:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 581.185605] Lustre: server umount lustre-MDT0001 complete [ 594.396922] Lustre: server umount lustre-OST0000 complete [ 597.489194] Lustre: 3662:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1763334896/real 1763334896] req@ffff89e8fbf50000 x1848990139377664/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1763334912 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 597.515538] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 597.539265] Lustre: Skipped 2 previous similar messages [ 597.785248] Lustre: server umount lustre-OST0001 complete [ 610.587238] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing load_modules_local [ 617.489976] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 617.727068] LustreError: 28167:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 620.704302] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing set_default_debug -1 all [ 626.894735] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 629.570734] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing set_default_debug -1 all [ 631.470556] Lustre: 29283:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 635.762940] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 640.257802] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing set_default_debug -1 all [ 647.095816] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 647.499297] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 647.519124] Lustre: Skipped 6 previous similar messages [ 654.122608] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing set_default_debug -1 all [ 662.290693] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 665.500988] Lustre: 31144:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 678.373018] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 678.381679] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 678.382066] Lustre: Skipped 3 previous similar messages [ 678.383581] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 678.414606] Lustre: Skipped 9 previous similar messages [ 681.013590] LustreError: 31345:0:(obd_class.h:479:obd_check_dev()) Device 10 not setup [ 681.018038] LustreError: 31345:0:(obd_class.h:479:obd_check_dev()) Skipped 25 previous similar messages [ 681.067653] Lustre: server umount lustre-MDT0000 complete [ 683.492676] LustreError: 28167:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 683.509714] LustreError: 28167:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 8 previous similar messages [ 683.790412] LustreError: 29285:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763334998 with bad export cookie 5223382813477267033 [ 683.792818] LustreError: MGC192.168.202.117@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 683.803620] LustreError: 29285:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 684.059469] Lustre: server umount lustre-MDT0001 complete [ 696.901455] Lustre: server umount lustre-OST0000 complete [ 710.423858] Lustre: server umount lustre-OST0001 complete [ 718.593765] Lustre: DEBUG MARKER: == conf-sanity test 76b: verify params log setup correctly ========================================================== 18:17:12 (1763335032) [ 743.667651] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing load_modules_local [ 753.615123] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 753.993954] LustreError: 33929:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 757.756172] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing set_default_debug -1 all [ 764.769420] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 768.197684] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing set_default_debug -1 all [ 770.591483] Lustre: 35043:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 775.620269] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 780.893765] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing set_default_debug -1 all [ 789.020937] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 794.464792] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing set_default_debug -1 all [ 801.054897] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 804.040985] Lustre: 36898:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 816.103372] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 816.112916] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 816.113783] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 816.128429] Lustre: Skipped 3 previous similar messages [ 816.135059] Lustre: Skipped 2 previous similar messages [ 819.377771] LustreError: 37053:0:(obd_class.h:479:obd_check_dev()) Device 10 not setup [ 819.381672] LustreError: 37053:0:(obd_class.h:479:obd_check_dev()) Skipped 25 previous similar messages [ 819.497763] Lustre: server umount lustre-MDT0000 complete [ 822.557208] LustreError: 33908:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763335137 with bad export cookie 5223382813477268139 [ 822.561459] LustreError: MGC192.168.202.117@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 822.574284] LustreError: 33908:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 822.986554] Lustre: server umount lustre-MDT0001 complete [ 835.651938] Lustre: server umount lustre-OST0000 complete [ 849.440303] Lustre: server umount lustre-OST0001 complete [ 855.928525] Lustre: DEBUG MARKER: == conf-sanity test 76c: verify changelog_mask is applied with lctl set_param -P ========================================================== 18:19:29 (1763335169) [ 865.021931] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing load_modules_local [ 871.878278] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 874.579292] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing set_default_debug -1 all [ 880.650289] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 883.891110] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing set_default_debug -1 all [ 886.368967] Lustre: 40137:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 892.046826] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 896.487155] LustreError: 40494:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 896.500369] LustreError: 40494:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 11 previous similar messages [ 897.457746] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing set_default_debug -1 all [ 905.332235] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 905.586448] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 905.596845] Lustre: Skipped 7 previous similar messages [ 910.420414] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing set_default_debug -1 all [ 915.956192] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 918.515037] Lustre: 41992:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 924.150338] Lustre: Setting parameter general.mdd.*.changelog_mask=-CLOSE in log params [ 924.152773] Lustre: Skipped 1 previous similar message [ 932.439428] Lustre: server umount lustre-MDT0000 complete [ 935.411602] LustreError: 42048:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763335250 with bad export cookie 5223382813477269238 [ 935.415968] LustreError: MGC192.168.202.117@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 941.736505] Lustre: server umount lustre-MDT0001 complete [ 948.355056] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 960.992637] LustreError: 3659:0:(client.c:1390:ptlrpc_import_delay_req()) @@@ invalidate in flight req@ffff89e7c12a2300 x1848990139525632/t0(0) o250->MGC192.168.202.117@tcp@0@lo:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl Rpc:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 965.683293] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing set_default_debug -1 all [ 970.602361] Lustre: lustre-MDT0000-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 973.112612] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 976.419426] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing set_default_debug -1 all [ 978.935569] Lustre: lustre-MDT0001-lwp-OST0001: Connection restored to 0@lo (at 0@lo) [ 978.947503] Lustre: Skipped 1 previous similar message [ 980.707155] Lustre: DEBUG MARKER: oleg217-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 984.196996] Lustre: DEBUG MARKER: mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 2 sec [ 986.977058] Lustre: DEBUG MARKER: oleg217-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 988.743482] Lustre: DEBUG MARKER: mdc.lustre-MDT0001-mdc-*.mds_server_uuid in FULL state after 0 sec [ 993.766930] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 993.770874] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 993.782410] Lustre: Skipped 6 previous similar messages [ 993.785477] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 993.791564] Lustre: Skipped 14 previous similar messages [ 998.559852] Lustre: server umount lustre-MDT0000 complete [ 1001.199786] LustreError: 42048:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763335315 with bad export cookie 5223382813477270435 [ 1001.204068] LustreError: 42048:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 1001.207890] LustreError: MGC192.168.202.117@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1001.390987] Lustre: server umount lustre-MDT0001 complete [ 1013.798692] Lustre: server umount lustre-OST0000 complete [ 1026.971168] Lustre: server umount lustre-OST0001 complete [ 1033.893884] Lustre: DEBUG MARKER: == conf-sanity test 76d: verify llite.*.xattr_cache can be set by 'lctl set_param -P' correctly ========================================================== 18:22:27 (1763335347) [ 1046.941843] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing load_modules_local [ 1054.561722] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1057.673831] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing set_default_debug -1 all [ 1063.470365] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1066.440669] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing set_default_debug -1 all [ 1068.726383] Lustre: 47487:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1073.386687] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1077.782787] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing set_default_debug -1 all [ 1084.832177] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1089.763535] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing set_default_debug -1 all [ 1095.398991] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1103.621179] Lustre: Setting parameter general.llite.*.xattr_cache=0 in log params [ 1115.107595] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1118.004866] LustreError: 49563:0:(obd_class.h:479:obd_check_dev()) Device 10 not setup [ 1118.012673] LustreError: 49563:0:(obd_class.h:479:obd_check_dev()) Skipped 67 previous similar messages [ 1118.121249] Lustre: server umount lustre-MDT0000 complete [ 1121.347357] LustreError: 46347:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763335435 with bad export cookie 5223382813477271219 [ 1121.357806] LustreError: MGC192.168.202.117@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1121.638766] Lustre: server umount lustre-MDT0001 complete [ 1134.630923] Lustre: server umount lustre-OST0000 complete [ 1147.975796] Lustre: server umount lustre-OST0001 complete [ 1157.013975] Lustre: DEBUG MARKER: == conf-sanity test 77: comma-separated MGS NIDs and failover node NIDs ========================================================== 18:24:30 (1763335470) [ 1159.214627] Lustre: DEBUG MARKER: SKIP: conf-sanity test_77 mixed loopback and real device not working [ 1160.819218] Lustre: DEBUG MARKER: == conf-sanity test 78: run resize2fs on MDT and OST filesystems ========================================================== 18:24:34 (1763335474) [ 1169.238323] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing set_hostid [ 1176.382594] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing load_modules_local [ 1184.525780] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1189.879364] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1195.580827] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1195.665523] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1195.873245] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1195.946309] Lustre: lustre-MDT0000: new disk, initializing [ 1196.063087] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1199.196323] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing set_default_debug -1 all [ 1205.650387] Lustre: DEBUG MARKER: oleg217-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1210.833156] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1210.888186] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1212.800612] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 1212.828167] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 1215.215909] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing set_default_debug -1 all [ 1221.163898] Lustre: DEBUG MARKER: oleg217-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1223.048746] Lustre: DEBUG MARKER: create test files [ 1253.862047] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1253.862300] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1253.882622] Lustre: Skipped 7 previous similar messages [ 1253.888399] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1253.891378] Lustre: Skipped 9 previous similar messages [ 1255.278554] Lustre: server umount lustre-OST0000 complete [ 1258.785585] Lustre: server umount lustre-MDT0000 complete [ 1267.970531] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing unload_modules_local [ 1270.584218] Key type lgssc unregistered [ 1270.909886] LNet: 54900:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1270.917257] LNetError: 54900:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1270.934597] LNet: Removed LNI 192.168.202.117@tcp [ 1271.640329] Key type .llcrypt unregistered [ 1271.642757] Key type ._llcrypt unregistered [ 1292.581252] Key type ._llcrypt registered [ 1292.583778] Key type .llcrypt registered [ 1292.657866] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing load_modules_local [ 1293.084292] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1293.094028] alg: No test for adler32 (adler32-zlib) [ 1294.064795] Lustre: Lustre: Build Version: 2.16.61_54_g3c0e0e6 [ 1294.308822] LNet: Added LNI 192.168.202.117@tcp [8/256/0/180] [ 1296.001199] Key type lgssc registered [ 1296.771631] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1302.319698] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1302.339945] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1303.732137] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1306.582073] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing set_default_debug -1 all [ 1310.663690] Lustre: DEBUG MARKER: oleg217-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1315.886744] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1316.123188] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1320.293333] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing set_default_debug -1 all [ 1324.354859] Lustre: DEBUG MARKER: oleg217-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1329.398995] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:102 to 0x240000400:129) [ 1329.480246] Lustre: 57485:0:(osd_internal.h:1472:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 515 < left 544, rollback = 9 [ 1329.484754] Lustre: 57485:0:(osd_handler.c:2082:osd_trans_dump_creds()) create: 32/128/0, destroy: 0/0/0 [ 1329.489119] Lustre: 57485:0:(osd_handler.c:2089:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 1329.494269] Lustre: 57485:0:(osd_handler.c:2099:osd_trans_dump_creds()) write: 2/13/1, punch: 0/0/0, quota 1/3/0 [ 1329.498877] Lustre: 57485:0:(osd_handler.c:2106:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 1329.502537] Lustre: 57485:0:(osd_handler.c:2113:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 1336.483180] Lustre: DEBUG MARKER: check files after expanding the MDT and OST filesystems [ 1341.474702] Lustre: DEBUG MARKER: create more files after expanding the MDT and OST filesystems [ 1350.119792] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1350.124484] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1350.131630] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1352.202121] LustreError: 58255:0:(obd_class.h:479:obd_check_dev()) Device 13 not setup [ 1352.371671] Lustre: server umount lustre-OST0000 complete [ 1355.444797] LustreError: 58456:0:(obd_class.h:479:obd_check_dev()) Device 8 not setup [ 1355.456371] LustreError: 58456:0:(obd_class.h:479:obd_check_dev()) Skipped 1 previous similar message [ 1355.686732] Lustre: server umount lustre-MDT0000 complete [ 1363.752120] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing unload_modules_local [ 1365.890318] Key type lgssc unregistered [ 1366.137481] LNet: 58990:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1366.150072] LNetError: 58990:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1367.206880] LNet: Removed LNI 192.168.202.117@tcp [ 1367.879502] Key type .llcrypt unregistered [ 1367.881899] Key type ._llcrypt unregistered [ 1392.966413] Key type ._llcrypt registered [ 1392.969696] Key type .llcrypt registered [ 1393.080853] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing load_modules_local [ 1393.964280] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1393.973880] alg: No test for adler32 (adler32-zlib) [ 1394.961337] Lustre: Lustre: Build Version: 2.16.61_54_g3c0e0e6 [ 1395.213527] LNet: Added LNI 192.168.202.117@tcp [8/256/0/180] [ 1396.848197] Key type lgssc registered [ 1397.852364] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1404.778334] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1404.808820] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1406.473441] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1409.655976] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing set_default_debug -1 all [ 1413.227423] Lustre: DEBUG MARKER: oleg217-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1417.842445] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1418.169952] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1422.718947] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing set_default_debug -1 all [ 1427.178687] Lustre: DEBUG MARKER: oleg217-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1431.415205] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:140 to 0x240000400:161) [ 1431.511141] Lustre: 61693:0:(osd_internal.h:1472:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 515 < left 544, rollback = 9 [ 1431.521866] Lustre: 61693:0:(osd_handler.c:2082:osd_trans_dump_creds()) create: 32/128/0, destroy: 0/0/0 [ 1431.529674] Lustre: 61693:0:(osd_handler.c:2089:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 1431.537802] Lustre: 61693:0:(osd_handler.c:2099:osd_trans_dump_creds()) write: 2/13/1, punch: 0/0/0, quota 1/3/0 [ 1431.547143] Lustre: 61693:0:(osd_handler.c:2106:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 1431.552633] Lustre: 61693:0:(osd_handler.c:2113:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 1438.160757] Lustre: DEBUG MARKER: check files after shrinking the MDT and OST filesystems [ 1446.881811] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1446.883342] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1446.895462] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1452.012274] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1452.240971] LustreError: 62387:0:(obd_class.h:479:obd_check_dev()) Device 13 not setup [ 1452.354847] Lustre: server umount lustre-OST0000 complete [ 1456.953464] LustreError: 62591:0:(obd_class.h:479:obd_check_dev()) Device 8 not setup [ 1456.958216] LustreError: 62591:0:(obd_class.h:479:obd_check_dev()) Skipped 1 previous similar message [ 1457.194476] Lustre: server umount lustre-MDT0000 complete [ 1467.853514] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing unload_modules_local [ 1470.648868] Key type lgssc unregistered [ 1470.907534] LNet: 63123:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1470.917139] LNetError: 63123:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1470.935971] LNet: Removed LNI 192.168.202.117@tcp [ 1472.033369] Key type .llcrypt unregistered [ 1472.034750] Key type ._llcrypt unregistered [ 1488.509424] Key type ._llcrypt registered [ 1488.511387] Key type .llcrypt registered [ 1488.620809] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing set_hostid [ 1502.338919] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing load_modules_local [ 1503.324542] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1503.352348] alg: No test for adler32 (adler32-zlib) [ 1504.390674] Lustre: Lustre: Build Version: 2.16.61_54_g3c0e0e6 [ 1504.570488] LNet: Added LNI 192.168.202.117@tcp [8/256/0/180] [ 1506.256318] Key type lgssc registered [ 1507.138097] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1515.472772] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1520.851866] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1525.393148] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1530.099975] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1534.954291] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1534.998445] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1535.005791] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1536.172586] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 1536.205448] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1536.251663] Lustre: lustre-MDT0000: new disk, initializing [ 1536.326336] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1536.343100] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1539.280738] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing set_default_debug -1 all [ 1547.801241] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1547.859913] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1547.928517] Lustre: 66250:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 1547.979706] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 1547.985751] Lustre: Skipped 1 previous similar message [ 1548.032862] Lustre: lustre-MDT0001: new disk, initializing [ 1548.116064] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 1548.158708] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 1548.164530] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 1551.396433] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing set_default_debug -1 all [ 1557.338988] Lustre: DEBUG MARKER: oleg217-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1559.993808] Lustre: DEBUG MARKER: oleg217-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 1565.786536] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1565.836827] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1566.123930] Lustre: lustre-OST0000: new disk, initializing [ 1566.134658] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1566.221892] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1570.726797] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing set_default_debug -1 all [ 1575.492305] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 1575.498706] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 1575.572556] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 1576.090679] Lustre: DEBUG MARKER: oleg217-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1578.798518] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 1578.926477] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 1581.920271] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 1582.110391] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 1586.148196] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1586.156636] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1586.165843] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1586.168671] Lustre: Skipped 1 previous similar message [ 1589.689084] LustreError: 69187:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 1589.755800] Lustre: server umount lustre-OST0000 complete [ 1594.339797] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1594.344083] LustreError: Skipped 1 previous similar message [ 1594.347272] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1594.354705] Lustre: Skipped 1 previous similar message [ 1594.357550] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1596.385796] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1596.389306] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1598.583277] LustreError: 69389:0:(obd_class.h:479:obd_check_dev()) Device 22 not setup [ 1598.590776] LustreError: 69389:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 1598.687262] Lustre: server umount lustre-MDT0000 complete [ 1601.356371] LustreError: 66240:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763335915 with bad export cookie 1457343026227905031 [ 1601.358395] LustreError: MGC192.168.202.117@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1601.360587] LustreError: 66240:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1601.442425] LustreError: 69591:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 1601.445267] LustreError: 69591:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 1601.567810] Lustre: server umount lustre-MDT0001 complete [ 1607.248652] Lustre: DEBUG MARKER: == conf-sanity test 79: format MDT/OST without mgs option (should return errors) ========================================================== 18:32:01 (1763335921) [ 1614.040582] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing load_modules_local [ 1622.886445] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1642.716915] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing set_hostid [ 1649.275781] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing load_modules_local [ 1657.152265] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1662.351090] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1666.609709] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1671.266476] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1676.483338] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1676.526540] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1676.676550] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 1676.704208] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1676.761732] Lustre: lustre-MDT0000: new disk, initializing [ 1676.835259] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1676.848636] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1679.904038] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing set_default_debug -1 all [ 1687.481762] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1687.537749] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1687.610272] Lustre: 73828:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 1687.663580] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 1687.668110] Lustre: Skipped 1 previous similar message [ 1687.718443] Lustre: lustre-MDT0001: new disk, initializing [ 1687.813520] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 1687.847879] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 1687.856839] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 1690.475926] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing set_default_debug -1 all [ 1696.285877] Lustre: DEBUG MARKER: oleg217-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1698.966664] Lustre: DEBUG MARKER: oleg217-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 1703.645302] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1703.689740] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1703.898809] Lustre: lustre-OST0000: new disk, initializing [ 1703.901884] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1703.973029] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1708.106814] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing set_default_debug -1 all [ 1713.708395] Lustre: DEBUG MARKER: oleg217-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1713.734357] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 1713.748804] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 1713.849588] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 1716.807735] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 1716.999548] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 1720.126244] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 1720.272757] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 1724.391173] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1724.398803] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1724.407524] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1724.420801] Lustre: Skipped 1 previous similar message [ 1724.429594] Lustre: Skipped 2 previous similar messages [ 1727.752373] LustreError: 76769:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 1727.762272] LustreError: 76769:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 1727.853367] Lustre: server umount lustre-OST0000 complete [ 1734.116591] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1734.121458] LustreError: Skipped 1 previous similar message [ 1734.123653] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1734.146469] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1736.725295] LustreError: 76972:0:(obd_class.h:479:obd_check_dev()) Device 22 not setup [ 1736.734972] LustreError: 76972:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 1736.844214] Lustre: server umount lustre-MDT0000 complete [ 1740.353203] LustreError: 73820:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763336054 with bad export cookie 1457343026227905913 [ 1740.355620] LustreError: MGC192.168.202.117@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1740.362825] LustreError: 73820:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1740.619510] Lustre: server umount lustre-MDT0001 complete [ 1747.221823] Lustre: DEBUG MARKER: == conf-sanity test 80: mgc import reconnect race ======== 18:34:20 (1763336060) [ 1753.156772] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1753.504740] LustreError: 77845:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1753.519724] LustreError: 77845:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 1753.565485] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1756.829187] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing set_default_debug -1 all [ 1758.693829] LustreError: 77845:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1763.815114] LustreError: 77845:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1764.815722] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1768.567898] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing set_default_debug -1 all [ 1773.272734] Lustre: DEBUG MARKER: oleg217-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1776.023137] Lustre: DEBUG MARKER: oleg217-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 1781.859213] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1786.700911] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing set_default_debug -1 all [ 1790.813215] Lustre: DEBUG MARKER: oleg217-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1793.857868] Lustre: 80181:0:(genops.c:1791:obd_export_evict_by_uuid()) MGS: evicting 558b0ac2-46fa-4b52-856c-db60681510a0 at adminstrative request [ 1795.560721] LustreError: MGC192.168.202.117@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1795.566889] Lustre: *** cfs_fail_loc=906, val=2147483648*** [ 1805.794710] Lustre: MGS: Client 558b0ac2-46fa-4b52-856c-db60681510a0 (at 0@lo) reconnecting [ 1805.800170] Lustre: Evicted from MGS (at 0@lo) after server handle changed from 0x143985c2d23508eb to 0x143985c2d2350b84 [ 1805.811675] Lustre: MGC192.168.202.117@tcp: Connection restored to 0@lo (at 0@lo) [ 1830.284794] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1830.359280] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1830.478577] Lustre: lustre-OST0001: new disk, initializing [ 1830.484683] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 1830.566167] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 1830.575766] Lustre: Skipped 2 previous similar messages [ 1835.070267] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing set_default_debug -1 all [ 1837.159936] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 1837.174621] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 1837.297914] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 1841.719975] Lustre: DEBUG MARKER: oleg217-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid [ 1847.782410] LustreError: lustre-OST0001-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1847.786180] Lustre: lustre-OST0001-osc-MDT0000: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1847.788346] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 1847.794927] Lustre: Skipped 2 previous similar messages [ 1847.798586] Lustre: Skipped 1 previous similar message [ 1850.497864] LustreError: 81316:0:(obd_class.h:479:obd_check_dev()) Device 25 not setup [ 1850.500087] LustreError: 81316:0:(obd_class.h:479:obd_check_dev()) Skipped 13 previous similar messages [ 1850.572646] Lustre: server umount lustre-OST0001 complete [ 1852.904694] LustreError: 80194:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1852.935723] LustreError: 80194:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 1856.994800] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 1856.998291] LustreError: Skipped 2 previous similar messages [ 1858.032260] LustreError: 79441:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1858.039679] LustreError: 79441:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 1859.877973] Lustre: server umount lustre-OST0000 complete [ 1867.746889] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1867.768859] Lustre: Skipped 2 previous similar messages [ 1867.779324] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1867.789964] Lustre: Skipped 5 previous similar messages [ 1869.566586] LustreError: 81720:0:(obd_class.h:479:obd_check_dev()) Device 28 not setup [ 1869.570677] LustreError: 81720:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 1869.633070] Lustre: server umount lustre-MDT0000 complete [ 1873.064236] LustreError: 77824:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763336187 with bad export cookie 1457343026227907460 [ 1873.065131] LustreError: MGC192.168.202.117@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1873.079817] LustreError: 77824:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 1873.400373] Lustre: server umount lustre-MDT0001 complete [ 1879.951436] Lustre: DEBUG MARKER: == conf-sanity test 81: sparse OST indexing ============== 18:36:33 (1763336193) [ 1881.222147] Lustre: DEBUG MARKER: SKIP: conf-sanity test_81 needs >= 3 OSTs [ 1882.844901] Lustre: DEBUG MARKER: == conf-sanity test 82a: specify OSTs for file (succeed) or directory (succeed) ========================================================== 18:36:36 (1763336196) [ 1884.422109] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82a needs >= 3 OSTs [ 1886.246919] Lustre: DEBUG MARKER: == conf-sanity test 82b: specify OSTs for file with --pool and --ost-list options ========================================================== 18:36:39 (1763336199) [ 1887.665220] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82b needs >= 4 OSTs [ 1889.422147] Lustre: DEBUG MARKER: == conf-sanity test 83: ENOSPACE on OST doesn't cause message VFS: Busy inodes after unmount ... ========================================================== 18:36:42 (1763336202) [ 1890.874697] Lustre: DEBUG MARKER: mount the OST /dev/mapper/ost1_flakey as a ldiskfs filesystem [ 1895.824988] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1896.898639] loop: module loaded [ 1896.922178] loop0: detected capacity change from 0 to 4294967296 [ 1896.974084] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 1898.710980] Lustre: DEBUG MARKER: run llverfs in partial mode on the OST ldiskfs /mnt/lustre-ost1 [ 1901.138180] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing run_llverfs /mnt/lustre-ost1 -vpl no [ 1904.856258] Lustre: DEBUG MARKER: unmount the OST /dev/mapper/ost1_flakey [ 1911.921127] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1915.522961] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1919.130306] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1922.716732] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1929.984285] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1930.042676] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1930.065801] LustreError: 84497:0:(obd_config.c:783:class_setup()) setup lustre-OST0000-osd failed (-28) [ 1930.077543] LustreError: 84497:0:(obd_mount.c:193:lustre_start_simple()) lustre-OST0000-osd setup error -28 [ 1930.085309] LustreError: 84497:0:(tgt_mount.c:2447:server_fill_super()) Unable to start osd on /dev/mapper/ost1_flakey: -28 [ 1930.088249] LustreError: 84497:0:(super25.c:199:lustre_fill_super()) llite: Unable to mount : rc = -28 [ 1948.594758] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing set_hostid [ 1957.438901] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing load_modules_local [ 1967.304318] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1973.428879] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1978.786295] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1983.907650] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1989.922935] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1989.977534] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1990.144886] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 1990.173373] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1990.237342] Lustre: lustre-MDT0000: new disk, initializing [ 1990.356887] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1990.384397] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1993.691266] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing set_default_debug -1 all [ 2002.966120] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2003.033832] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2003.112210] Lustre: 87344:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2003.330047] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2006.226518] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing set_default_debug -1 all [ 2012.756372] Lustre: DEBUG MARKER: oleg217-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2016.092708] Lustre: DEBUG MARKER: oleg217-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2022.100981] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2022.170465] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2027.804623] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing set_default_debug -1 all [ 2030.151823] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2030.283949] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2034.478352] Lustre: DEBUG MARKER: oleg217-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2037.747831] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2037.898797] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2041.054470] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2041.215160] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2045.921119] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2045.926838] LustreError: Skipped 1 previous similar message [ 2045.933735] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2045.951922] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2045.959305] Lustre: Skipped 2 previous similar messages [ 2045.963448] Lustre: Skipped 2 previous similar messages [ 2057.184137] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2057.243128] LustreError: 90282:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 2057.299967] LustreError: 90282:0:(obd_class.h:479:obd_check_dev()) Skipped 17 previous similar messages [ 2057.415322] Lustre: server umount lustre-OST0000 complete [ 2075.104130] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2075.295668] Lustre: server umount lustre-MDT0000 complete [ 2076.650065] LustreError: 90285:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2076.669586] LustreError: 90285:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 2078.283399] LustreError: 89088:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763336392 with bad export cookie 1457343026227908062 [ 2078.287466] LustreError: MGC192.168.202.117@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2078.295344] LustreError: 89088:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2078.533273] Lustre: server umount lustre-MDT0001 complete [ 2086.124851] Lustre: DEBUG MARKER: == conf-sanity test 84: check recovery_hard_time ========= 18:39:59 (1763336399) [ 2092.683763] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2093.153094] LustreError: 91360:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2096.779617] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing set_default_debug -1 all [ 2104.507598] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2107.886713] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing set_default_debug -1 all [ 2112.612168] Lustre: DEBUG MARKER: oleg217-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2116.134855] Lustre: DEBUG MARKER: oleg217-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2122.574563] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2123.039691] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2123.042853] Lustre: Skipped 4 previous similar messages [ 2127.204533] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing set_default_debug -1 all [ 2132.352946] Lustre: DEBUG MARKER: oleg217-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2135.901611] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2139.455618] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 3 sec [ 2143.916263] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2144.121943] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2151.461845] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2151.573866] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2151.759519] Lustre: lustre-OST0001: new disk, initializing [ 2151.761203] Lustre: Skipped 2 previous similar messages [ 2151.766717] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 2151.769563] Lustre: Skipped 3 previous similar messages [ 2156.588985] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing set_default_debug -1 all [ 2157.108647] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 2157.112323] Lustre: Skipped 2 previous similar messages [ 2157.114821] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 2157.210440] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 2163.391096] Lustre: DEBUG MARKER: oleg217-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid [ 2166.660457] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid 50 [ 2166.823877] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2170.334988] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [ 2170.550399] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2186.857661] Lustre: DEBUG MARKER: mds1 REPLAY BARRIER on lustre-MDT0000 [ 2210.989715] Lustre: Failing over lustre-MDT0000 [ 2211.263944] LustreError: 96289:0:(obd_class.h:479:obd_check_dev()) Device 28 not setup [ 2211.267297] LustreError: 96289:0:(obd_class.h:479:obd_check_dev()) Skipped 17 previous similar messages [ 2211.351284] Lustre: server umount lustre-MDT0000 complete [ 2212.324443] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2212.334282] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2212.352480] Lustre: Skipped 2 previous similar messages [ 2212.355959] LustreError: 91359:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2212.374126] LustreError: 91359:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 2229.713603] Lustre: 64461:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1763336528/real 1763336528] req@00000000d85116d9 x1848991619985152/t0(0) o400->MGC192.168.202.117@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1763336544 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 2229.740644] LustreError: MGC192.168.202.117@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2234.067388] LDISKFS-fs (dm-0): 4 truncates cleaned up [ 2234.069429] LDISKFS-fs (dm-0): recovery complete [ 2234.079309] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2239.970773] LustreError: 64459:0:(client.c:1390:ptlrpc_import_delay_req()) @@@ invalidate in flight req@00000000da5b01a5 x1848991619992320/t0(0) o250->MGC192.168.202.117@tcp@0@lo:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl Rpc:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 2240.205202] Lustre: lustre-MDT0000: in recovery but waiting for the first client to connect [ 2243.180867] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing set_default_debug -1 all [ 2244.228051] Lustre: lustre-MDT0000: Will be in recovery for at least 1:00, or until 3 clients reconnect [ 2244.229462] Lustre: 95291:0:(ldlm_lib.c:2067:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 2245.606531] Lustre: lustre-MDT0000-lwp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 2245.611583] Lustre: 92072:0:(ldlm_lib.c:2067:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 2245.615855] Lustre: 92072:0:(ldlm_lib.c:2067:extend_recovery_timer()) Skipped 1 previous similar message [ 2245.745780] LustreError: 97038:0:(ldlm_lib.c:2679:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2246.066861] LustreError: 97038:0:(ldlm_lib.c:2679:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2246.402768] LustreError: 97038:0:(ldlm_lib.c:2679:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2246.415706] LustreError: 97038:0:(ldlm_lib.c:2679:replay_request_or_update()) Skipped 1 previous similar message [ 2246.744159] LustreError: 97038:0:(ldlm_lib.c:2679:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2246.749665] LustreError: 97038:0:(ldlm_lib.c:2679:replay_request_or_update()) Skipped 1 previous similar message [ 2246.776544] Lustre: 97038:0:(ldlm_lib.c:2067:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2246.781056] Lustre: 97038:0:(ldlm_lib.c:2067:extend_recovery_timer()) Skipped 7 previous similar messages [ 2247.436607] LustreError: 97038:0:(ldlm_lib.c:2679:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2247.440835] LustreError: 97038:0:(ldlm_lib.c:2679:replay_request_or_update()) Skipped 2 previous similar messages [ 2247.760137] LustreError: 97038:0:(ldlm_lib.c:2679:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2247.769352] LustreError: 97038:0:(ldlm_lib.c:2679:replay_request_or_update()) Skipped 2 previous similar messages [ 2249.058419] Lustre: 97038:0:(ldlm_lib.c:2067:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2249.079594] Lustre: 97038:0:(ldlm_lib.c:2067:extend_recovery_timer()) Skipped 6 previous similar messages [ 2249.721619] LustreError: 97038:0:(ldlm_lib.c:2679:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2249.739893] LustreError: 97038:0:(ldlm_lib.c:2679:replay_request_or_update()) Skipped 6 previous similar messages [ 2250.066441] LustreError: 97038:0:(ldlm_lib.c:2679:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2250.070172] LustreError: 97038:0:(ldlm_lib.c:2679:replay_request_or_update()) Skipped 6 previous similar messages [ 2253.234856] Lustre: 97038:0:(ldlm_lib.c:2067:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2253.245072] Lustre: 97038:0:(ldlm_lib.c:2067:extend_recovery_timer()) Skipped 12 previous similar messages [ 2253.875690] LustreError: 97038:0:(ldlm_lib.c:2679:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2253.884794] LustreError: 97038:0:(ldlm_lib.c:2679:replay_request_or_update()) Skipped 12 previous similar messages [ 2254.201389] LustreError: 97038:0:(ldlm_lib.c:2679:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2254.210679] LustreError: 97038:0:(ldlm_lib.c:2679:replay_request_or_update()) Skipped 12 previous similar messages [ 2261.250042] Lustre: 97038:0:(ldlm_lib.c:2067:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2261.262302] Lustre: 97038:0:(ldlm_lib.c:2067:extend_recovery_timer()) Skipped 24 previous similar messages [ 2261.917817] LustreError: 97038:0:(ldlm_lib.c:2679:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2261.936811] LustreError: 97038:0:(ldlm_lib.c:2679:replay_request_or_update()) Skipped 24 previous similar messages [ 2262.264174] LustreError: 97038:0:(ldlm_lib.c:2679:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2262.272188] LustreError: 97038:0:(ldlm_lib.c:2679:replay_request_or_update()) Skipped 24 previous similar messages [ 2277.300118] Lustre: 97038:0:(ldlm_lib.c:2067:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2277.309459] Lustre: 97038:0:(ldlm_lib.c:2067:extend_recovery_timer()) Skipped 49 previous similar messages [ 2277.969444] LustreError: 97038:0:(ldlm_lib.c:2679:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2277.992283] LustreError: 97038:0:(ldlm_lib.c:2679:replay_request_or_update()) Skipped 49 previous similar messages [ 2278.320183] LustreError: 97038:0:(ldlm_lib.c:2679:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2278.324074] LustreError: 97038:0:(ldlm_lib.c:2679:replay_request_or_update()) Skipped 49 previous similar messages [ 2304.192402] Lustre: 97038:0:(ldlm_lib.c:2375:target_recovery_overseer()) lustre-MDT0000 recovery is aborted by hard timeout [ 2304.197638] Lustre: 97038:0:(ldlm_lib.c:2385:target_recovery_overseer()) recovery is aborted, evict exports in recovery [ 2304.203377] Lustre: 97038:0:(genops.c:1618:class_disconnect_stale_exports()) lustre-MDT0000: disconnect stale client 64f66f90-6026-444a-8858-c8a275b28576@192.168.202.17@tcp [ 2304.212411] Lustre: lustre-MDT0000: disconnecting 1 stale clients [ 2304.219096] Lustre: 97038:0:(ldlm_lib.c:1897:abort_req_replay_queue()) @@@ aborted: req@00000000e71bcf24 x1848991606182784/t0(8589934779) o101->64f66f90-6026-444a-8858-c8a275b28576@192.168.202.17@tcp:154/0 lens 592/0 e 0 to 0 dl 1763336629 ref 1 fl Complete:/604/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:0 [ 2304.248381] Lustre: lustre-MDT0000-osd: cancel update llog [0x200000400:0x1:0x0] [ 2304.260024] Lustre: lustre-MDT0000: Denying connection for new client 64f66f90-6026-444a-8858-c8a275b28576 (at 192.168.202.17@tcp), waiting for 3 known clients (2 recovered, 0 in progress, and 1 evicted) already passed deadline 0:00 [ 2304.297881] Lustre: lustre-MDT0001-osp-MDT0000: cancel update llog [0x240000401:0x1:0x0] [ 2304.344277] Lustre: lustre-MDT0000-osp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 2304.347407] Lustre: Skipped 2 previous similar messages [ 2304.354929] Lustre: lustre-MDT0000: Recovery over after 1:00, of 3 clients 2 recovered and 1 was evicted. [ 2304.439208] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:79 to 0x280000401:577) [ 2304.441840] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:48 to 0x2c0000401:545) [ 2314.731100] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2314.741124] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2314.744306] Lustre: Skipped 11 previous similar messages [ 2320.753456] Lustre: server umount lustre-OST0000 complete [ 2322.405791] LustreError: 94761:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2322.440751] LustreError: 94761:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 38 previous similar messages [ 2330.884628] Lustre: server umount lustre-OST0001 complete [ 2341.138299] LustreError: 98102:0:(obd_class.h:479:obd_check_dev()) Device 10 not setup [ 2341.140556] LustreError: 98102:0:(obd_class.h:479:obd_check_dev()) Skipped 15 previous similar messages [ 2341.278253] Lustre: server umount lustre-MDT0000 complete [ 2344.571451] LustreError: 94384:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763336659 with bad export cookie 1457343026227959281 [ 2344.582808] LustreError: MGC192.168.202.117@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2344.886533] Lustre: server umount lustre-MDT0001 complete [ 2352.076423] Lustre: DEBUG MARKER: == conf-sanity test 85: osd_ost init: fail ea_fid_set ==== 18:44:25 (1763336665) [ 2358.284664] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2358.289776] Lustre: *** cfs_fail_loc=197, val=0*** [ 2373.792606] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 2378.912446] LustreError: 98995:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.202.117@tcp: failed processing log, type 4: rc = -110 [ 2384.081152] Lustre: *** cfs_fail_loc=197, val=0*** [ 2384.089917] Lustre: Skipped 65 previous similar messages [ 2404.640324] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2404.643473] Lustre: Skipped 2 previous similar messages [ 2408.781509] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing set_default_debug -1 all [ 2414.160622] Lustre: DEBUG MARKER: oleg217-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2415.990335] Lustre: server umount lustre-OST0000 complete [ 2422.476972] Lustre: DEBUG MARKER: == conf-sanity test 86: Replacing mkfs.lustre -G option == 18:45:36 (1763336736) [ 2442.313563] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2447.578153] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing set_default_debug -1 all [ 2452.962571] LustreError: 100504:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2452.971443] LustreError: 100504:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 6 previous similar messages [ 2454.595412] Lustre: DEBUG MARKER: oleg217-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2459.004541] LustreError: 100534:0:(osp_object.c:617:osp_attr_get()) lustre-MDT0001-osp-MDT0000: osp_attr_get update error [0x200000009:0x1:0x0]: rc = -5 [ 2459.009621] LustreError: 100534:0:(lod_sub_object.c:917:lod_sub_prep_llog()) lustre-MDT0000-mdtlov: can't get id from catalogs: rc = -5 [ 2459.014030] LustreError: 100534:0:(lod_dev.c:510:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 16, retries 0, failed: rc = -5 [ 2459.372666] Lustre: server umount lustre-MDT0000 complete [ 2472.080866] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2484.765453] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing set_hostid [ 2490.982919] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing load_modules_local [ 2497.556526] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2502.533174] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2508.070387] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2513.550501] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2518.981814] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2519.061525] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2519.349507] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2519.378688] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2519.438697] Lustre: lustre-MDT0000: new disk, initializing [ 2519.589264] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2522.932436] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing set_default_debug -1 all [ 2531.039318] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2531.089162] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2531.148275] Lustre: 104572:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2531.397045] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2534.401640] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing set_default_debug -1 all [ 2540.355215] Lustre: DEBUG MARKER: oleg217-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2543.082805] Lustre: DEBUG MARKER: oleg217-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2548.776728] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2548.832167] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2553.411079] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing set_default_debug -1 all [ 2558.914409] Lustre: DEBUG MARKER: oleg217-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2559.110542] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2562.077271] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2562.254206] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2565.400450] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2565.610027] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2569.188086] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2569.194057] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2569.200032] Lustre: Skipped 10 previous similar messages [ 2569.206952] Lustre: Skipped 11 previous similar messages [ 2573.201863] Lustre: server umount lustre-OST0000 complete [ 2577.386978] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2577.392071] LustreError: Skipped 4 previous similar messages [ 2590.177156] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2590.416466] Lustre: server umount lustre-MDT0000 complete [ 2594.878759] LustreError: 107713:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763336909 with bad export cookie 1457343026227969935 [ 2594.882493] LustreError: MGC192.168.202.117@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2594.890577] LustreError: 107713:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 8 previous similar messages [ 2595.410698] Lustre: server umount lustre-MDT0001 complete [ 2603.693961] Lustre: DEBUG MARKER: == conf-sanity test 87: check if MDT inode can hold EAs with N stripes properly ========================================================== 18:48:37 (1763336917) [ 2610.392659] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing unload_modules_local [ 2612.650276] Key type lgssc unregistered [ 2612.918605] LNet: 108644:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2612.926141] LNetError: 108644:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2612.947186] LNet: Removed LNI 192.168.202.117@tcp [ 2613.583153] Key type .llcrypt unregistered [ 2613.584754] Key type ._llcrypt unregistered [ 2627.520341] Key type ._llcrypt registered [ 2627.521956] Key type .llcrypt registered [ 2627.621361] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing set_hostid [ 2639.059371] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing load_modules_local [ 2639.821310] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2639.943822] alg: No test for adler32 (adler32-zlib) [ 2640.924647] Lustre: Lustre: Build Version: 2.16.61_54_g3c0e0e6 [ 2641.095649] LNet: Added LNI 192.168.202.117@tcp [8/256/0/180] [ 2642.744325] Key type lgssc registered [ 2643.585033] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2651.530347] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2658.326951] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2664.268872] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2669.789256] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2674.647718] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2680.258923] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2680.314215] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 2680.334249] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2681.563329] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2681.601889] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2681.665887] Lustre: lustre-MDT0000: new disk, initializing [ 2681.728754] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2681.742728] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2684.332602] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing set_default_debug -1 all [ 2693.143460] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2693.210658] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2693.394905] Lustre: lustre-OST0000: new disk, initializing [ 2693.404366] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 2693.410153] Lustre: Skipped 1 previous similar message [ 2693.543455] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2696.983944] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 2696.994630] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 2697.025970] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 2698.167519] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing set_default_debug -1 all [ 2706.391611] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2706.443917] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2706.548549] Lustre: lustre-OST0001: new disk, initializing [ 2706.559484] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 2706.634249] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 2710.756832] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing set_default_debug -1 all [ 2712.125230] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:1:ost [ 2712.129992] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:1:ost] [ 2712.164991] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x280000400 [ 2717.153974] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2717.154738] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2717.159273] Lustre: Skipped 1 previous similar message [ 2717.166052] Lustre: Skipped 1 previous similar message [ 2722.274725] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2722.277238] Lustre: Skipped 1 previous similar message [ 2723.059237] LustreError: 114201:0:(obd_class.h:479:obd_check_dev()) Device 17 not setup [ 2723.154704] Lustre: server umount lustre-MDT0000 complete [ 2728.296346] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: (null) [ 2732.152252] LustreError: 111959:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763337046 with bad export cookie 15343469327035257388 [ 2732.161716] LustreError: MGC192.168.202.117@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2732.167198] LustreError: 111959:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 2732.213527] LustreError: 114796:0:(obd_class.h:479:obd_check_dev()) Device 12 not setup [ 2732.240830] LustreError: 114796:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 2732.290062] Lustre: server umount lustre-OST0000 complete [ 2734.834086] LustreError: 114998:0:(obd_class.h:479:obd_check_dev()) Device 2 not setup [ 2734.841497] LustreError: 114998:0:(obd_class.h:479:obd_check_dev()) Skipped 1 previous similar message [ 2734.944458] Lustre: server umount lustre-OST0001 complete [ 2740.579318] Lustre: DEBUG MARKER: == conf-sanity test 88: check the default mount options can be overridden ========================================================== 18:50:54 (1763337054) [ 2748.197862] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing load_modules_local [ 2754.410774] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2755.218474] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2759.492380] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2760.341686] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2765.578515] Lustre: DEBUG MARKER: == conf-sanity test 89: check tunefs --param and --erase-param{s} options ========================================================== 18:51:19 (1763337079) [ 2775.833956] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2777.592511] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2779.310854] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2780.884775] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2781.632062] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2783.259747] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2796.477261] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing set_hostid [ 2803.585900] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing load_modules_local [ 2811.027283] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2816.347960] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2821.290625] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2825.819281] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2831.040272] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2831.094540] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2831.286530] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2831.305558] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2831.355377] Lustre: lustre-MDT0000: new disk, initializing [ 2831.406914] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2831.420040] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2834.220528] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing set_default_debug -1 all [ 2842.363246] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2842.426906] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2842.485972] Lustre: 120332:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2842.529834] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 2842.532429] Lustre: Skipped 1 previous similar message [ 2842.589857] Lustre: lustre-MDT0001: new disk, initializing [ 2842.689647] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2842.709276] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 2842.720392] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2845.660611] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing set_default_debug -1 all [ 2850.844680] Lustre: DEBUG MARKER: oleg217-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2853.252382] Lustre: DEBUG MARKER: oleg217-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2857.575704] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2857.627422] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2857.789371] Lustre: lustre-OST0000: new disk, initializing [ 2857.792366] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 2857.835148] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2861.552686] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing set_default_debug -1 all [ 2864.700177] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 2864.709218] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2864.791307] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2867.834632] Lustre: DEBUG MARKER: oleg217-client.virtnet: executing wait_import_state_mount (FULL|IDLE) failover.node=192.0.2.254@tcp0,osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2870.947664] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2871.061292] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2873.449523] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2873.583726] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2875.365220] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2875.365426] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2875.368112] LustreError: Skipped 1 previous similar message [ 2875.368772] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2875.373821] Lustre: Skipped 1 previous similar message [ 2875.386461] Lustre: Skipped 1 previous similar message [ 2880.482150] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2880.490812] Lustre: Skipped 1 previous similar message [ 2885.607823] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2889.184365] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2889.233432] LustreError: 123271:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 2889.238393] LustreError: 123271:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 2889.348028] Lustre: server umount lustre-OST0000 complete [ 2893.793555] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2893.812186] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2893.831325] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2893.840258] Lustre: Skipped 1 previous similar message [ 2895.842852] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2898.740658] LustreError: 123475:0:(obd_class.h:479:obd_check_dev()) Device 22 not setup [ 2898.749398] LustreError: 123475:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 2898.872469] Lustre: server umount lustre-MDT0000 complete [ 2902.045429] LustreError: 120325:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763337216 with bad export cookie 15343469327035258613 [ 2902.056269] LustreError: MGC192.168.202.117@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2902.057879] LustreError: 120325:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2902.323184] Lustre: server umount lustre-MDT0001 complete [ 2910.438592] Lustre: DEBUG MARKER: == conf-sanity test 90a: check max_mod_rpcs_in_flight is enforced ========================================================== 18:53:44 (1763337224) [ 2916.451053] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2916.822793] LustreError: 124348:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2916.838897] LustreError: 124348:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 2916.901553] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2920.401902] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing set_default_debug -1 all [ 2921.956934] LustreError: 124348:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2927.076769] LustreError: 124348:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2928.017930] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2931.879114] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing set_default_debug -1 all [ 2936.534745] Lustre: DEBUG MARKER: oleg217-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2939.577163] Lustre: DEBUG MARKER: oleg217-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2945.027133] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2949.631931] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing set_default_debug -1 all [ 2954.827677] Lustre: DEBUG MARKER: oleg217-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2970.178173] Lustre: *** cfs_fail_loc=159, val=0*** [ 2986.677902] Lustre: lustre-MDT0000: Client 2d1c1c5f-f5c6-4e1d-8649-45536c69e30b (at 192.168.202.17@tcp) reconnecting [ 2987.804821] Lustre: *** cfs_fail_loc=159, val=0*** [ 2987.807677] Lustre: Skipped 5 previous similar messages [ 3003.031556] Lustre: lustre-MDT0000: Client 2d1c1c5f-f5c6-4e1d-8649-45536c69e30b (at 192.168.202.17@tcp) reconnecting [ 3023.330747] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3023.331250] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3023.345403] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3023.355670] Lustre: Skipped 3 previous similar messages [ 3026.687111] LustreError: 126864:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 3026.690470] LustreError: 126864:0:(obd_class.h:479:obd_check_dev()) Skipped 13 previous similar messages [ 3026.812882] Lustre: server umount lustre-OST0000 complete [ 3031.015163] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3033.573567] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3033.579443] Lustre: Skipped 2 previous similar messages [ 3044.320419] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 3044.472446] LustreError: 127065:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup [ 3044.489408] LustreError: 127065:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 3044.587117] Lustre: server umount lustre-MDT0000 complete [ 3048.555457] LustreError: 124328:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763337363 with bad export cookie 15343469327035259495 [ 3048.562158] LustreError: MGC192.168.202.117@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3048.568956] LustreError: 124328:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 3049.017191] Lustre: server umount lustre-MDT0001 complete [ 3060.429304] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing unload_modules_local [ 3062.091908] Key type lgssc unregistered [ 3062.296575] LNet: 127799:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3062.300431] LNetError: 127799:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3062.314514] LNet: Removed LNI 192.168.202.117@tcp [ 3062.849194] Key type .llcrypt unregistered [ 3062.851047] Key type ._llcrypt unregistered [ 3080.811873] Key type ._llcrypt registered [ 3080.813231] Key type .llcrypt registered [ 3080.913924] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing load_modules_local [ 3081.561883] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3081.679715] alg: No test for adler32 (adler32-zlib) [ 3082.682908] Lustre: Lustre: Build Version: 2.16.61_54_g3c0e0e6 [ 3082.860173] LNet: Added LNI 192.168.202.117@tcp [8/256/0/180] [ 3084.536235] Key type lgssc registered [ 3085.731518] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3092.122917] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3092.139987] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3093.509429] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3096.137266] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing set_default_debug -1 all [ 3103.343309] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3103.654327] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3107.080940] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing set_default_debug -1 all [ 3111.129209] Lustre: DEBUG MARKER: oleg217-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 3114.143095] Lustre: DEBUG MARKER: oleg217-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 3120.762386] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3121.014506] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3125.780087] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing set_default_debug -1 all [ 3130.193911] Lustre: DEBUG MARKER: oleg217-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3139.457113] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:42 to 0x280000401:65) [ 3149.022600] Lustre: *** cfs_fail_loc=159, val=0*** [ 3165.350314] Lustre: lustre-MDT0000: Client 4a2b9e73-11a3-4c1f-bcba-1733ce5c8133 (at 192.168.202.17@tcp) reconnecting [ 3167.554804] Lustre: *** cfs_fail_loc=159, val=0*** [ 3167.559450] Lustre: Skipped 1 previous similar message [ 3183.810917] Lustre: lustre-MDT0001: Client 4a2b9e73-11a3-4c1f-bcba-1733ce5c8133 (at 192.168.202.17@tcp) reconnecting [ 3185.116199] Lustre: *** cfs_fail_loc=159, val=0*** [ 3185.118022] Lustre: Skipped 2 previous similar messages [ 3201.692178] Lustre: lustre-MDT0001: Client 4a2b9e73-11a3-4c1f-bcba-1733ce5c8133 (at 192.168.202.17@tcp) reconnecting [ 3205.518614] Lustre: 131898:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 3207.964340] Lustre: *** cfs_fail_loc=159, val=0*** [ 3207.966411] Lustre: Skipped 4 previous similar messages [ 3223.195338] Lustre: lustre-MDT0000: Client f0b2b256-c8cd-4031-8322-ae2b1d903ba3 (at 192.168.202.17@tcp) reconnecting [ 3224.918346] Lustre: *** cfs_fail_loc=159, val=0*** [ 3224.920316] Lustre: Skipped 14 previous similar messages [ 3240.120576] Lustre: lustre-MDT0000: Client f0b2b256-c8cd-4031-8322-ae2b1d903ba3 (at 192.168.202.17@tcp) reconnecting [ 3241.518674] Lustre: 132183:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 3262.435014] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3262.439996] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3262.451385] LustreError: Skipped 1 previous similar message [ 3262.457847] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3262.463918] Lustre: Skipped 1 previous similar message [ 3266.695181] LustreError: 132307:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 3266.829091] Lustre: server umount lustre-OST0000 complete [ 3272.673386] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3272.678058] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3272.694291] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3272.713628] Lustre: Skipped 2 previous similar messages [ 3276.387186] LustreError: 132509:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup [ 3276.390118] LustreError: 132509:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 3276.531346] Lustre: server umount lustre-MDT0000 complete [ 3277.794291] LustreError: 129174:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3277.811474] LustreError: 129174:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 3279.979250] LustreError: 129154:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763337594 with bad export cookie 15490357563486049215 [ 3279.980775] LustreError: MGC192.168.202.117@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3280.004986] LustreError: 129154:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 1 previous similar message [ 3280.203943] LustreError: 132714:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [ 3280.206740] LustreError: 132714:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 3280.377740] Lustre: server umount lustre-MDT0001 complete [ 3290.797405] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing unload_modules_local [ 3293.552405] Key type lgssc unregistered [ 3293.860668] LNet: 133245:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3293.874291] LNetError: 133245:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3293.889702] LNet: Removed LNI 192.168.202.117@tcp [ 3294.626449] Key type .llcrypt unregistered [ 3294.630696] Key type ._llcrypt unregistered [ 3316.185531] Key type ._llcrypt registered [ 3316.186918] Key type .llcrypt registered [ 3316.269127] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing load_modules_local [ 3317.218990] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3317.272386] alg: No test for adler32 (adler32-zlib) [ 3318.356085] Lustre: Lustre: Build Version: 2.16.61_54_g3c0e0e6 [ 3318.551241] LNet: Added LNI 192.168.202.117@tcp [8/256/0/180] [ 3320.216163] Key type lgssc registered [ 3321.335384] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3327.917420] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3327.936082] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3329.217251] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3332.532036] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing set_default_debug -1 all [ 3334.625138] LustreError: 134657:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3334.641768] LustreError: 134657:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 3338.726204] LustreError: 134657:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3340.960734] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3341.422720] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3345.708619] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing set_default_debug -1 all [ 3351.740401] Lustre: DEBUG MARKER: oleg217-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 3355.187992] Lustre: DEBUG MARKER: oleg217-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 3362.243428] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3362.745890] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3365.811275] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:97) [ 3368.245427] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing set_default_debug -1 all [ 3374.034979] Lustre: DEBUG MARKER: oleg217-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3374.061748] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:129) [ 3381.305491] Lustre: 136919:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 3384.739815] Lustre: 136967:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 3386.573798] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 in log params [ 3387.469036] Lustre: Setting parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=8 in log params [ 3390.632212] Lustre: 137116:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify lustre/mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 (mode = 0) failed: rc = -17 [ 3391.678554] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=7 in log params [ 3392.665181] Lustre: 137215:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 3396.578907] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3396.588994] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3396.616822] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3399.656593] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3399.662950] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3399.668577] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3402.316133] LustreError: 137323:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 3402.538911] Lustre: server umount lustre-OST0000 complete [ 3407.842879] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3407.881394] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3407.914052] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3407.917162] Lustre: Skipped 1 previous similar message [ 3409.890084] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3409.924921] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3409.927452] Lustre: Skipped 1 previous similar message [ 3415.015965] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3421.152396] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 3421.264648] LustreError: 137525:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup [ 3421.270107] LustreError: 137525:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 3421.364755] Lustre: server umount lustre-MDT0000 complete [ 3425.014756] LustreError: 137019:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763337739 with bad export cookie 3151328950027236849 [ 3425.018680] LustreError: MGC192.168.202.117@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3425.028792] LustreError: 137019:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 3425.155479] LustreError: 137728:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [ 3425.175031] LustreError: 137728:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 3425.436482] Lustre: server umount lustre-MDT0001 complete [ 3435.101744] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing unload_modules_local [ 3438.260582] Key type lgssc unregistered [ 3438.573612] LNet: 138260:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3438.586824] LNetError: 138260:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3438.612851] LNet: Removed LNI 192.168.202.117@tcp [ 3439.460817] Key type .llcrypt unregistered [ 3439.463226] Key type ._llcrypt unregistered [ 3460.793443] Key type ._llcrypt registered [ 3460.797912] Key type .llcrypt registered [ 3460.914133] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing load_modules_local [ 3461.888445] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3461.979499] alg: No test for adler32 (adler32-zlib) [ 3463.054766] Lustre: Lustre: Build Version: 2.16.61_54_g3c0e0e6 [ 3463.250981] LNet: Added LNI 192.168.202.117@tcp [8/256/0/180] [ 3464.888266] Key type lgssc registered [ 3465.909360] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3472.589331] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3472.610483] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3474.020698] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3477.850314] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing set_default_debug -1 all [ 3484.133997] LustreError: 139649:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3484.163936] LustreError: 139649:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 3485.425855] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3485.784544] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3489.306392] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing set_default_debug -1 all [ 3493.645918] Lustre: DEBUG MARKER: oleg217-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 3496.521598] Lustre: DEBUG MARKER: oleg217-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 3502.518422] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3502.859937] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3507.275829] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing set_default_debug -1 all [ 3511.091678] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:129) [ 3511.796945] Lustre: DEBUG MARKER: oleg217-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3519.292921] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:161) [ 3527.615595] Lustre: *** cfs_fail_loc=159, val=0*** [ 3527.626422] Lustre: Skipped 1 previous similar message [ 3543.253794] Lustre: lustre-MDT0000: Client f15ae977-265a-4f2e-9bfb-cf1998c16a51 (at 192.168.202.17@tcp) reconnecting [ 3549.179279] LustreError: lustre-OST0000-osc-MDT0000: operation ost_destroy to node 0@lo failed: rc = -107 [ 3549.182323] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3549.206885] LustreError: Skipped 1 previous similar message [ 3549.223215] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3550.098116] LustreError: 142044:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 3550.229354] Lustre: server umount lustre-OST0000 complete [ 3555.305077] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3555.305981] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3555.318625] Lustre: Skipped 1 previous similar message [ 3559.702357] LustreError: 142245:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup [ 3559.705351] LustreError: 142245:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 3559.823821] Lustre: server umount lustre-MDT0000 complete [ 3560.419638] LustreError: 139643:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3560.447978] LustreError: 139643:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 3563.403041] LustreError: 139629:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763337878 with bad export cookie 14489499372569354898 [ 3563.407852] LustreError: MGC192.168.202.117@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3563.420757] LustreError: 139629:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 1 previous similar message [ 3573.742300] LustreError: 142447:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [ 3573.748723] LustreError: 142447:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 3573.933162] Lustre: server umount lustre-MDT0001 complete [ 3584.741255] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing unload_modules_local [ 3587.004415] Key type lgssc unregistered [ 3587.310118] LNet: 142982:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3587.326198] LNetError: 142982:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3587.354306] LNet: Removed LNI 192.168.202.117@tcp [ 3588.180317] Key type .llcrypt unregistered [ 3588.181891] Key type ._llcrypt unregistered [ 3614.125195] Key type ._llcrypt registered [ 3614.126770] Key type .llcrypt registered [ 3614.283124] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing load_modules_local [ 3615.300850] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3615.349151] alg: No test for adler32 (adler32-zlib) [ 3616.662458] Lustre: Lustre: Build Version: 2.16.61_54_g3c0e0e6 [ 3616.929432] LNet: Added LNI 192.168.202.117@tcp [8/256/0/180] [ 3618.616221] Key type lgssc registered [ 3619.777727] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3628.854767] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3628.869884] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3630.411676] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3634.919234] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing set_default_debug -1 all [ 3639.777885] LustreError: 144378:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3639.792097] LustreError: 144378:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 3643.471496] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3643.971394] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3648.492973] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing set_default_debug -1 all [ 3654.554435] Lustre: DEBUG MARKER: oleg217-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 3658.180908] Lustre: DEBUG MARKER: oleg217-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 3664.796960] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3665.261191] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3669.364735] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:161) [ 3670.380165] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing set_default_debug -1 all [ 3674.504881] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:170 to 0x280000401:193) [ 3676.360904] Lustre: DEBUG MARKER: oleg217-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3695.874761] Lustre: lustre-OST0000: evicting e226975c-237f-4779-9a58-e521fd8e698b (at 192.168.202.17@tcp) by administrative request [ 3725.805114] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3725.808968] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3725.825507] Lustre: Skipped 1 previous similar message [ 3725.827446] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3729.606264] LustreError: 147164:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 3729.704932] Lustre: server umount lustre-OST0000 complete [ 3736.035151] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3736.037863] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3736.039650] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3736.039658] Lustre: Skipped 1 previous similar message [ 3736.062485] Lustre: Skipped 1 previous similar message [ 3736.120072] LustreError: Skipped 1 previous similar message [ 3739.389047] LustreError: 147365:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup [ 3739.398983] LustreError: 147365:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 3739.563931] Lustre: server umount lustre-MDT0000 complete [ 3741.166715] LustreError: 145097:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3741.192957] LustreError: 145097:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 3742.826642] LustreError: 144360:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763338057 with bad export cookie 2384766475423752862 [ 3742.836098] LustreError: MGC192.168.202.117@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3742.951559] LustreError: 147566:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [ 3742.954952] LustreError: 147566:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 3743.127927] Lustre: server umount lustre-MDT0001 complete [ 3752.092478] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing unload_modules_local [ 3754.756597] Key type lgssc unregistered [ 3755.031843] LNet: 148098:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3755.037522] LNetError: 148098:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3755.047705] LNet: Removed LNI 192.168.202.117@tcp [ 3755.722343] Key type .llcrypt unregistered [ 3755.724032] Key type ._llcrypt unregistered [ 3822.286710] Key type ._llcrypt registered [ 3822.300983] Key type .llcrypt registered [ 3822.428735] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing load_modules_local [ 3823.267365] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3823.289329] alg: No test for adler32 (adler32-zlib) [ 3824.329486] Lustre: Lustre: Build Version: 2.16.61_54_g3c0e0e6 [ 3824.529642] LNet: Added LNI 192.168.202.117@tcp [8/256/0/180] [ 3826.168390] Key type lgssc registered [ 3827.202292] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3834.224595] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3834.238425] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3835.607880] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3838.790350] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing set_default_debug -1 all [ 3840.995157] LustreError: 150565:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3841.007070] LustreError: 150565:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 3846.118825] LustreError: 150565:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3846.346697] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3846.736043] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3850.537684] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing set_default_debug -1 all [ 3856.184817] Lustre: DEBUG MARKER: oleg217-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 3859.502990] Lustre: DEBUG MARKER: oleg217-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 3865.630846] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3866.010734] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3871.128828] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing set_default_debug -1 all [ 3872.184051] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:193) [ 3875.781650] Lustre: DEBUG MARKER: oleg217-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3881.402798] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:195 to 0x280000401:225) [ 3887.588212] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3887.601031] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3887.620664] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3891.684789] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3891.687287] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3891.689108] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3891.700602] Lustre: Skipped 1 previous similar message [ 3896.807305] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3896.811561] Lustre: Skipped 1 previous similar message [ 3900.897884] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 3900.928043] LustreError: 152820:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 3901.029925] Lustre: server umount lustre-OST0000 complete [ 3907.044680] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3907.053322] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3910.291299] LustreError: 153023:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup [ 3910.307217] LustreError: 153023:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 3910.410391] Lustre: server umount lustre-MDT0000 complete [ 3912.161433] LustreError: 151594:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3912.175786] LustreError: 151594:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 3913.644529] LustreError: 150545:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763338228 with bad export cookie 4593204024267505603 [ 3913.658474] LustreError: MGC192.168.202.117@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3913.674065] LustreError: 150545:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 3913.788159] LustreError: 153224:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [ 3913.843358] LustreError: 153224:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 3914.078419] Lustre: server umount lustre-MDT0001 complete [ 3923.092723] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing unload_modules_local [ 3925.623782] Key type lgssc unregistered [ 3925.898862] LNet: 153756:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3925.904817] LNetError: 153756:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3925.917594] LNet: Removed LNI 192.168.202.117@tcp [ 3926.652341] Key type .llcrypt unregistered [ 3926.655206] Key type ._llcrypt unregistered [ 3946.480335] Key type ._llcrypt registered [ 3946.482203] Key type .llcrypt registered [ 3946.571162] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing load_modules_local [ 3947.154503] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3947.167212] alg: No test for adler32 (adler32-zlib) [ 3948.124633] Lustre: Lustre: Build Version: 2.16.61_54_g3c0e0e6 [ 3948.264991] LNet: Added LNI 192.168.202.117@tcp [8/256/0/180] [ 3949.929350] Key type lgssc registered [ 3950.806739] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3956.187472] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3961.917877] Lustre: DEBUG MARKER: == conf-sanity test 100: check lshowmount lists MGS, MDT, OST and 0@lo ========================================================== 19:11:16 (1763338276) [ 3974.055802] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing set_hostid [ 3981.515971] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing load_modules_local [ 3988.962870] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3994.916459] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4000.148200] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4006.641380] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4012.880687] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4012.922373] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4012.934157] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4014.118233] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 4014.144521] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 4014.201307] Lustre: lustre-MDT0000: new disk, initializing [ 4014.297307] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4014.318154] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 4017.221784] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing set_default_debug -1 all [ 4025.411631] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4025.466319] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4025.542119] Lustre: 157958:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 4025.592661] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 4025.595864] Lustre: Skipped 1 previous similar message [ 4025.672459] Lustre: lustre-MDT0001: new disk, initializing [ 4025.771695] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4025.807176] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 4025.816233] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 4028.575164] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing set_default_debug -1 all [ 4034.498549] Lustre: DEBUG MARKER: oleg217-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 4038.027454] Lustre: DEBUG MARKER: oleg217-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 4044.322073] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4044.411731] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4044.642865] Lustre: lustre-OST0000: new disk, initializing [ 4044.649231] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 4044.743518] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4049.816633] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing set_default_debug -1 all [ 4051.015451] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 4051.019158] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 4051.159231] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 4056.251384] Lustre: DEBUG MARKER: oleg217-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4066.786899] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4066.789130] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4066.791544] LustreError: Skipped 1 previous similar message [ 4066.792191] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4066.831540] Lustre: Skipped 1 previous similar message [ 4069.960820] LustreError: 160546:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 4070.100096] Lustre: server umount lustre-OST0000 complete [ 4077.030685] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4077.031893] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4077.041517] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4077.041525] Lustre: Skipped 1 previous similar message [ 4077.054665] Lustre: Skipped 1 previous similar message [ 4079.797496] LustreError: 160748:0:(obd_class.h:479:obd_check_dev()) Device 22 not setup [ 4079.809702] LustreError: 160748:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 4079.915153] Lustre: server umount lustre-MDT0000 complete [ 4083.105463] LustreError: 157949:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763338397 with bad export cookie 17225335406527406155 [ 4083.109724] LustreError: MGC192.168.202.117@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4083.120662] LustreError: 157949:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 4083.232302] LustreError: 160950:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 4083.235296] LustreError: 160950:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 4083.386183] Lustre: server umount lustre-MDT0001 complete [ 4091.557265] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing unload_modules_local [ 4093.434345] Key type lgssc unregistered [ 4093.663236] LNet: 161481:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4093.667238] LNetError: 161481:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4093.682709] LNet: Removed LNI 192.168.202.117@tcp [ 4094.188088] Key type .llcrypt unregistered [ 4094.189663] Key type ._llcrypt unregistered [ 4114.291874] Key type ._llcrypt registered [ 4114.293563] Key type .llcrypt registered [ 4114.395395] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing load_modules_local [ 4115.183273] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4115.283778] alg: No test for adler32 (adler32-zlib) [ 4116.310592] Lustre: Lustre: Build Version: 2.16.61_54_g3c0e0e6 [ 4116.503903] LNet: Added LNI 192.168.202.117@tcp [8/256/0/180] [ 4118.144512] Key type lgssc registered [ 4118.986375] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4124.784625] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4124.797691] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4126.092162] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4129.268980] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing set_default_debug -1 all [ 4135.394231] LustreError: 162876:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4135.412139] LustreError: 162876:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 4136.029560] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4136.307238] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4139.922864] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing set_default_debug -1 all [ 4145.229310] Lustre: DEBUG MARKER: oleg217-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 4148.579853] Lustre: DEBUG MARKER: oleg217-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 4154.304460] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4154.584643] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4160.027412] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing set_default_debug -1 all [ 4164.609768] Lustre: DEBUG MARKER: oleg217-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4170.999479] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:3 to 0x280000401:33) [ 4181.565791] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4181.611619] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4181.645620] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4181.669048] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4181.710357] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4182.862253] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4182.885462] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4182.905625] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4182.910600] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4182.921229] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4184.213647] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4184.237493] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4184.260417] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4184.268775] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4184.294158] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4185.249666] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4186.271570] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4186.274255] Lustre: Skipped 1 previous similar message [ 4186.297454] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4186.303629] Lustre: Skipped 1 previous similar message [ 4186.309527] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4186.311767] Lustre: Skipped 1 previous similar message [ 4186.318497] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4186.327435] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4186.330151] Lustre: Skipped 1 previous similar message [ 4187.134312] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4188.142797] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4189.202723] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4190.224969] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4191.125336] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4191.129637] Lustre: Skipped 4 previous similar messages [ 4191.155899] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4191.162983] Lustre: Skipped 4 previous similar messages [ 4191.166145] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4191.168272] Lustre: Skipped 4 previous similar messages [ 4191.174662] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4191.187232] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4191.199544] Lustre: Skipped 4 previous similar messages [ 4192.313722] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4193.401691] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4194.354047] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4195.454790] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4196.193867] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4197.074570] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4197.878826] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4198.802859] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4199.742184] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4199.759043] Lustre: Skipped 8 previous similar messages [ 4199.773395] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4199.779776] Lustre: Skipped 8 previous similar messages [ 4199.798579] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4199.807512] Lustre: Skipped 8 previous similar messages [ 4199.815455] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4199.830752] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4199.843505] Lustre: Skipped 8 previous similar messages [ 4200.735537] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4201.700153] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4202.702803] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4203.574357] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4204.475257] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4205.320292] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4205.905741] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4206.617758] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4207.701640] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4208.464386] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4209.443185] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4210.233554] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4211.065975] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4212.033079] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4212.792146] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4213.583878] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4214.368687] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4215.260929] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4215.989899] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4215.993308] Lustre: Skipped 18 previous similar messages [ 4216.006431] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4216.011355] Lustre: Skipped 18 previous similar messages [ 4216.019210] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4216.022201] Lustre: Skipped 18 previous similar messages [ 4216.030755] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4216.046332] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4216.048949] Lustre: Skipped 18 previous similar messages [ 4216.909765] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4217.606922] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4218.408746] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4219.240274] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4220.076921] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4220.925417] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4221.794649] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4222.599581] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4223.434895] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4224.208970] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4224.998436] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4225.832835] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4226.730994] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4227.503974] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4228.335493] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4229.017667] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4229.804667] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4230.685047] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4231.667948] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4232.435752] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4233.126354] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4233.795810] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4234.639351] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4235.417649] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4236.104259] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4236.837943] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4237.587054] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4238.328199] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4239.056827] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4239.739346] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4240.453790] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4241.111763] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4241.860778] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4242.553355] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4243.243127] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4243.842935] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4244.443173] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4245.153761] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4245.927422] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4246.660648] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4247.423585] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4248.161736] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4248.167616] Lustre: Skipped 41 previous similar messages [ 4248.199123] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4248.220334] Lustre: Skipped 41 previous similar messages [ 4248.224471] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4248.227565] Lustre: Skipped 41 previous similar messages [ 4248.230935] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4248.240513] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4248.243194] Lustre: Skipped 41 previous similar messages [ 4249.114599] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4249.831421] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4250.575590] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4251.209338] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4251.946533] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4252.610487] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4253.297275] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4253.920161] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4254.600397] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4255.197717] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4255.881167] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4256.575266] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4257.249621] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4257.888482] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4258.496625] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4259.228089] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4259.947410] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4260.706443] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4261.397785] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4262.052775] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4262.907338] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4263.581807] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4264.408354] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4265.103111] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4265.876948] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4266.776430] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4267.492236] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4268.197409] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4268.979136] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4269.706480] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4270.450562] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4271.180443] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4271.960319] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4272.683188] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4273.520107] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4274.271695] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4274.973378] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4275.716060] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4276.474610] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4277.236358] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4277.942285] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4278.798653] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4279.597822] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4280.269556] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4280.978075] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4281.699126] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4282.437273] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4283.213557] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4283.876621] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4284.573882] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4285.360791] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4286.094623] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4286.821236] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4287.622051] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4288.433442] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4289.301806] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4289.964491] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4290.717499] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4291.360899] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4292.191769] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4293.016872] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4293.771295] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4294.525938] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4295.302966] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4296.051040] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4296.839212] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4297.604795] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4298.535747] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4299.189247] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4300.055694] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4300.987876] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4301.979929] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4302.750462] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4303.773976] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4304.599512] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4305.391653] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4306.275505] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4307.230790] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4308.112077] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4309.075299] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4309.975562] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4310.784590] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4311.568381] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4312.379579] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4312.388682] Lustre: Skipped 83 previous similar messages [ 4312.404193] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4312.422079] Lustre: Skipped 83 previous similar messages [ 4312.431541] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4312.441443] Lustre: Skipped 83 previous similar messages [ 4312.448087] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4312.457643] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4312.460244] Lustre: Skipped 83 previous similar messages [ 4313.301871] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4314.164607] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4314.820693] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4315.687079] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4316.524447] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4317.395907] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4318.351221] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4319.129385] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4319.877723] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4320.713161] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4321.603757] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4322.404190] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4323.264622] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4324.024693] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4324.747267] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4325.747377] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4326.692555] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4327.581972] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4328.398639] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4329.269154] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4330.153880] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4331.079213] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4331.959546] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4332.661804] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4333.364903] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4334.099710] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4334.882906] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4335.548029] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4336.233438] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4336.939459] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4337.704295] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4338.468291] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4339.317363] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4340.176131] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4341.061555] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4341.927994] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4342.665910] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4343.360922] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4344.011618] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4344.692794] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4345.363555] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4346.026545] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4346.744174] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4347.338902] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4347.992856] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4348.695427] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4349.500678] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4350.179140] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4350.925331] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4351.764119] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4352.633182] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4353.430301] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4354.377620] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4355.225440] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4356.121413] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4357.149830] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4357.984249] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4358.988060] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4359.897611] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4360.812867] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4361.646437] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4362.615330] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4363.467892] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4364.340585] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4365.079547] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4365.807119] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4366.532623] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4367.369706] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4368.045787] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4368.932846] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4369.794764] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4370.626843] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4371.600615] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4372.457763] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4373.134760] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4373.960418] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4374.893737] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4375.781435] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4376.629475] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4377.572541] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4378.272919] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4379.194812] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4380.036627] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4380.799987] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4381.627031] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4382.531587] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4383.443190] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4384.380693] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4385.248842] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4386.117656] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4386.961341] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4387.859569] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4388.814566] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4389.688680] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4390.579422] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4391.437719] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4392.329976] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4393.363186] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4394.176886] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4395.042909] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4395.894203] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4396.667198] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4397.401854] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4398.112575] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4398.877421] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4399.584373] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4400.322305] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4400.991378] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4401.732118] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4402.455877] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4403.180230] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4403.994786] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4404.872209] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4405.638143] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4406.615582] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4407.361099] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4408.186577] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4408.827738] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4409.887624] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4410.832849] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4411.709439] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4412.768657] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4413.476880] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4414.256791] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4415.057668] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4415.992599] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4416.810942] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4417.623385] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4418.564212] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4419.297152] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4420.424640] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4421.326921] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4422.472982] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4423.456221] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4424.762802] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4425.697346] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4426.718649] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4427.708842] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4428.720772] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4429.580944] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4430.782841] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4431.654119] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4432.683406] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4433.617691] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4434.581791] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4435.489435] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4436.537693] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4437.381753] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4438.468940] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4439.234686] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4439.942408] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4440.601623] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4440.605488] Lustre: Skipped 151 previous similar messages [ 4440.615364] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4440.623888] Lustre: Skipped 151 previous similar messages [ 4440.630715] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4440.633308] Lustre: Skipped 151 previous similar messages [ 4440.639205] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4440.649386] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4440.652966] Lustre: Skipped 151 previous similar messages [ 4441.375415] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4442.131258] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4442.839439] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4443.468082] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4444.243289] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4444.946303] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4445.609966] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4446.337584] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4446.991543] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4447.568983] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4448.294876] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4448.948568] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4449.899723] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4450.715245] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4451.585456] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4452.240803] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4452.950319] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4453.629318] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4454.273889] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4454.986909] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4455.774713] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4456.464591] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4457.232663] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4458.001087] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4458.725264] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4459.563143] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4460.334730] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4461.117584] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4461.811228] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4462.410444] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4462.980475] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4463.711276] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4464.384631] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4464.954830] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4465.578117] LustreError: 162895:0:(osp_precreate.c:651:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 4465.583926] LustreError: 162895:0:(osp_precreate.c:1355:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 4465.592163] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4466.196191] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4466.667492] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4467.169986] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4467.711746] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4468.239979] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4468.909065] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4469.616272] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4470.165225] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4470.828600] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4471.541831] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4472.272804] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4473.094125] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4473.798616] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4474.484043] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4475.208600] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4476.015610] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4476.790257] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4477.501398] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4478.329254] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4479.193322] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4479.983714] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4480.809519] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4481.565394] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4482.214536] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4482.953438] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4483.889352] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4484.675881] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4485.513605] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4486.246289] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4486.974404] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4487.729355] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4488.532589] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4489.346563] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4490.168184] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4490.899933] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4491.581935] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4492.277437] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4492.938887] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4493.454483] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4494.041592] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4494.683784] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4495.336254] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4495.957972] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4496.584890] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4497.268598] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4497.965471] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4498.680109] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4499.271716] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4499.911775] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4500.596883] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4501.331813] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4502.019975] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4502.802987] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4503.539806] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4504.245886] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4504.908474] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4505.586960] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4506.267107] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4507.048178] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4507.791655] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4508.639275] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4509.394450] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4510.172978] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4510.896475] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4511.817524] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4512.464781] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4513.173396] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4513.846950] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4514.605944] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4515.365747] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4516.172469] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4516.935372] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4517.618775] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4518.270255] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4519.096636] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4519.815504] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4520.559547] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4521.442745] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4522.291361] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4522.993718] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4523.792120] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4524.578241] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4525.442553] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4526.383020] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4527.180976] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4528.050361] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4528.838351] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4529.598200] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4530.360630] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4531.181710] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4532.029537] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4532.677927] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4533.495478] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4534.381382] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4535.171932] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4535.894520] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4536.760513] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4537.775649] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4538.544857] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4539.333292] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4540.069418] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4540.798162] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4541.468100] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4542.239506] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4543.095215] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4543.809255] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4544.549735] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4545.276655] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4545.933446] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4546.649808] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4547.319890] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4547.991931] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4548.717466] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4549.477570] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4550.247761] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4550.969121] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4551.723585] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4552.414799] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4553.120437] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4553.870739] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4554.603753] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4555.271641] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4555.976872] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4556.733385] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4557.420483] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4558.176298] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4558.980946] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4559.805714] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4560.642610] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4561.537912] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4562.393191] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4563.228537] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4563.979885] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4564.852190] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4565.528684] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4566.281188] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4567.066604] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4567.836750] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4568.529578] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4569.228967] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4569.942216] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4570.801934] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4571.666853] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4572.519223] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4573.324561] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4574.251450] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4575.057855] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4575.908074] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4576.774180] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4577.597270] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4578.470474] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4579.261301] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4580.067092] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4580.825558] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4581.789487] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4582.623420] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4583.382606] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4584.297628] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4585.215376] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4586.056225] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4587.028617] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4587.979917] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4588.927395] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4589.844839] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4590.808292] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4591.762959] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4592.738397] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4593.482080] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4594.255915] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4594.970220] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4595.698779] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4596.539650] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4597.469123] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4598.361971] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4599.175371] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4599.966522] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4600.755203] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4601.529809] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4602.339990] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4603.236914] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4604.011253] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4604.766723] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4605.509222] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4606.167341] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4606.905817] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4607.597782] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4608.391077] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4609.166307] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4609.964969] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4610.731722] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4611.673606] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4612.436637] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4613.422486] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4614.225616] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4615.191762] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4616.002240] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4616.806548] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4617.618884] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4618.531823] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4619.316928] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4620.197222] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4620.991844] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4621.789421] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4622.697269] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4623.559964] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4624.525585] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4625.272140] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4626.128696] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4627.115200] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4628.030343] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4628.987749] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4629.785974] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4630.700925] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4631.631312] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4632.625294] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4633.493242] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4634.368538] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4635.280484] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4636.184287] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4637.023062] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4637.809909] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4638.659913] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4639.642116] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4640.515929] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4641.440927] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4642.387990] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4643.410970] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4644.165129] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4644.946844] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4645.793965] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4646.645340] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4647.570825] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4648.378771] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4649.081058] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4650.083391] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4650.951046] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4651.754786] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4652.514263] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4653.392540] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4654.225141] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4655.150906] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4656.059036] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4656.965600] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4657.820683] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4658.731031] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4659.484069] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4660.339074] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4661.253154] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4662.184414] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4662.987868] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4663.847274] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4664.692603] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4665.636091] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4666.763758] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4667.742663] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4668.654581] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4669.741630] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4670.861293] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4671.765017] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4672.747658] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4673.537668] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4674.437885] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4675.320571] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4676.319159] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4677.471069] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4678.359702] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4679.529161] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4680.309804] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4681.259813] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4682.120674] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4683.044194] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4683.900993] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4685.022770] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4685.974875] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4686.938589] LustreError: 162895:0:(osp_precreate.c:651:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 4686.944061] LustreError: 162895:0:(osp_precreate.c:1355:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 4686.954214] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4687.897556] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4688.911912] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4689.902936] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4691.161672] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4692.263380] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4693.674539] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4694.679375] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4695.758710] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4696.631850] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4696.651081] Lustre: Skipped 318 previous similar messages [ 4696.674169] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4696.688355] Lustre: Skipped 318 previous similar messages [ 4696.695499] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4696.704986] Lustre: Skipped 318 previous similar messages [ 4696.710512] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4696.718830] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4696.725091] Lustre: Skipped 318 previous similar messages [ 4697.757513] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4698.760712] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4699.712307] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4700.852536] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4701.889865] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4702.955426] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4704.061741] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4705.031624] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4706.051568] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4707.070838] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4708.018825] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4708.884573] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4709.769050] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4710.725172] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4711.787347] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4712.609649] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4713.518438] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4714.562648] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4715.366793] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4716.245096] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4717.162333] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4717.900425] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4718.723446] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4719.458735] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4720.180627] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4720.914654] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4721.675944] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4722.414791] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4723.106615] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4723.742894] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4724.386418] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4725.114759] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4725.851272] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4726.515068] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4727.341351] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4728.131376] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4728.934268] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4729.643771] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4730.473256] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4731.135603] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4731.820699] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4732.507498] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4733.208812] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4733.912974] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4734.743473] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4735.631322] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4736.382468] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4737.124460] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4737.967816] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4738.748027] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4739.548800] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4740.465475] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4741.254971] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4742.275314] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4743.295568] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4744.214267] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4745.127375] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4745.942465] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4746.747974] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4747.441652] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4748.208685] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4748.917831] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4749.853926] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4750.676079] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4751.401323] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4752.204234] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4752.961750] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4753.687469] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4754.581424] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4755.379651] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4756.221780] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4757.018363] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4757.991494] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4758.759899] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4759.808377] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4760.811938] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4761.668987] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4762.512440] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4763.270108] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4763.974347] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4764.715729] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4765.518971] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4766.208549] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4766.970102] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4767.739736] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4768.479980] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4769.337949] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4770.113025] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4770.931652] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4771.874151] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4772.714145] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4773.477608] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4774.219483] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4775.100482] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4775.922205] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4776.716557] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4777.480655] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4778.352024] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4779.114907] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4779.849332] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4780.787972] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4781.723755] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4782.624263] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4783.416372] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4784.251420] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4785.224546] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4786.119057] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4786.959413] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4787.737505] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4788.560934] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4789.441790] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4790.227725] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4791.101672] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4791.820970] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4792.560422] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4793.382271] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4794.131199] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4794.816886] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4795.487911] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4796.139844] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4796.773104] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4797.557225] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4798.272454] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4798.902847] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4799.598175] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4800.443311] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4801.239266] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4801.938602] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4802.612730] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4803.447592] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4804.179385] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4804.976304] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4805.658185] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4806.389699] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4807.053415] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4807.920373] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4808.758611] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4809.609803] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4810.335544] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4810.959242] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4811.723206] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4812.459686] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4813.154554] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4813.879139] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4814.646248] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4815.408922] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4816.110574] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4816.805406] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4817.572399] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4818.388869] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4819.163455] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4819.891847] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4820.699161] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4821.570262] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4822.316400] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4823.155498] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4823.892863] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4824.619233] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4825.359535] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4826.116645] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4826.941474] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4827.784456] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4828.528468] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4829.341817] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4830.111376] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4830.924898] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4831.636362] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4832.344435] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4832.966962] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4833.578707] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4834.267602] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4834.952074] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4835.606184] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4836.259085] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4836.922397] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4837.653153] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4838.384662] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4839.090303] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4839.754494] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4840.505341] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4841.267455] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4842.038517] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4842.833765] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4843.650367] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4844.637849] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4845.625297] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4846.496273] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4847.366673] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4848.366986] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4849.247869] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4850.071632] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4850.967499] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4851.931491] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4852.692771] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4853.602907] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4854.369751] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4855.290573] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4856.287381] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4857.168438] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4858.284117] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4859.117712] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4860.164444] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4861.237328] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4862.289051] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4863.281778] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4864.241232] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4865.279734] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4866.199267] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4867.105388] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4868.005380] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4868.975273] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4869.908349] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4870.781584] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4871.617587] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4872.478088] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4873.457673] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4874.368781] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4875.132143] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4875.882274] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4876.609617] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4877.527293] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4878.276985] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4879.144808] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4880.141492] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4881.085707] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4882.003727] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4882.943839] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4883.711412] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4884.690676] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4885.408657] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4886.375127] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4887.269954] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4888.213865] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4889.305444] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4890.587965] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4891.432609] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4892.533598] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4893.274639] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4894.279967] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4895.148273] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4895.921204] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4896.707896] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4897.660474] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4898.630900] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4899.375910] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4900.356343] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4901.109330] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4901.921796] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4902.608720] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4903.325249] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4904.101233] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4904.949985] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4905.734604] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4906.506956] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4907.156450] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4907.891421] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4908.531731] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4909.170990] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4909.746678] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4910.459895] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4911.205241] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4911.921451] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4912.746920] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4913.507158] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4914.225168] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4915.146484] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4915.939589] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4916.671038] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4917.483210] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4918.489295] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4919.151988] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4919.985740] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4920.912553] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4921.738539] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4922.585352] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4923.469349] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4924.323689] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4925.179529] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4926.076550] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4927.082666] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4927.826789] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4928.705687] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4929.529060] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4930.268468] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4931.126536] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4932.001459] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4932.848651] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4933.760725] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4934.577749] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4935.422205] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4936.406460] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4937.303201] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4938.182870] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4939.034197] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4939.863651] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4940.649867] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4941.517023] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4942.281659] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4943.054723] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4943.867396] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4944.635030] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4945.476709] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4946.225829] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4947.055737] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4948.020462] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4949.039163] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4949.972123] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4950.948658] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4951.908320] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4952.811365] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4953.715696] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4954.755943] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4955.571578] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4956.335460] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4957.186389] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4957.952633] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4958.642269] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4959.320364] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4960.125506] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4961.022243] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4961.883973] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4962.694582] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4963.584551] LustreError: 162895:0:(osp_precreate.c:651:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 4963.592921] LustreError: 162895:0:(osp_precreate.c:1355:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 4963.607995] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4964.393477] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4965.253340] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4965.990167] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4966.782249] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4967.494260] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4968.186910] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4968.907831] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4969.662751] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4970.395062] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4971.112326] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4971.781841] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4972.728679] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4973.501434] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4974.365853] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4975.092993] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4976.071681] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4977.128850] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4978.171467] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4979.135281] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4979.991706] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4980.798184] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4981.610819] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4982.493993] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4983.365446] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4984.053714] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4984.928106] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4986.022345] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4986.853049] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4987.891972] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4988.818312] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4989.775782] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4990.710923] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4991.971601] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4993.013468] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4993.979290] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4995.077124] LustreError: 162895:0:(osp_precreate.c:651:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 4995.095507] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4995.101434] LustreError: 162895:0:(osp_precreate.c:1355:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 4996.079846] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4997.111418] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4997.826700] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4998.587862] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4999.263116] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5000.003148] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5413.672548] LustreError: 165033:0:(ofd_dev.c:1780:ofd_create_hdl()) lustre-OST0000: unable to precreate: rc = -28 [ 5413.681798] LustreError: 162895:0:(osp_precreate.c:651:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -28 [ 6045.157430] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 6045.159388] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6045.161409] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6045.174024] Lustre: Skipped 366 previous similar messages [ 6049.546807] LustreError: 216845:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 6049.859207] Lustre: server umount lustre-OST0000 complete [ 6055.395050] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6055.408785] Lustre: Skipped 2 previous similar messages [ 6059.884929] LustreError: 217048:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup [ 6059.894536] LustreError: 217048:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 6060.119165] Lustre: server umount lustre-MDT0000 complete [ 6060.514229] LustreError: 162870:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6060.529937] LustreError: 162870:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 6063.411237] LustreError: 216645:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763340378 with bad export cookie 11267039156318472003 [ 6063.414195] LustreError: MGC192.168.202.117@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6063.433871] LustreError: 216645:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 6063.594754] LustreError: 217253:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [ 6063.598829] LustreError: 217253:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 6063.845922] Lustre: server umount lustre-MDT0001 complete [ 6072.708111] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing unload_modules_local [ 6075.265251] Key type lgssc unregistered [ 6075.526398] LNet: 217785:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 6075.530207] LNetError: 217785:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 6076.588710] LNet: Removed LNI 192.168.202.117@tcp [ 6077.345859] Key type .llcrypt unregistered [ 6077.347448] Key type ._llcrypt unregistered [ 6098.120906] Key type ._llcrypt registered [ 6098.122290] Key type .llcrypt registered [ 6098.246556] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing load_modules_local [ 6099.454822] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 6099.485365] alg: No test for adler32 (adler32-zlib) [ 6100.581857] Lustre: Lustre: Build Version: 2.16.61_54_g3c0e0e6 [ 6100.847720] LNet: Added LNI 192.168.202.117@tcp [8/256/0/180] [ 6102.544201] Key type lgssc registered [ 6103.897663] Lustre: Echo OBD driver; http://www.lustre.org/ [ 6112.266792] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 6112.291916] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6113.791976] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6117.888380] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing set_default_debug -1 all [ 6124.001966] LustreError: 219172:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6124.030139] LustreError: 219172:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 6126.128596] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 6126.490534] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 6129.934503] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing set_default_debug -1 all [ 6135.289749] Lustre: DEBUG MARKER: oleg217-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 6138.831740] Lustre: DEBUG MARKER: oleg217-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 6145.321844] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 6145.705307] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 6150.937558] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing set_default_debug -1 all [ 6156.705427] Lustre: DEBUG MARKER: oleg217-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 6159.126759] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49666 to 0x280000401:49697) [ 6165.922744] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 6167.795199] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6167.804619] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 6167.812538] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6167.822253] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 6167.823025] LustreError: 219192:0:(osp_precreate.c:1263:osp_precreate_thread()) cfs_fail_timeout id 2107 sleeping for 20000ms [ 6172.641174] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 6172.644131] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6172.658202] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6174.697603] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6174.699373] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6174.708103] Lustre: Skipped 1 previous similar message [ 6175.871546] LustreError: 221571:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 6176.058107] Lustre: server umount lustre-OST0000 complete [ 6187.832137] LustreError: 219192:0:(osp_precreate.c:1263:osp_precreate_thread()) cfs_fail_timeout id 2107 awake [ 6209.442922] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 6209.724322] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 6214.828253] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6214.845491] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 6215.992393] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing set_default_debug -1 all [ 6221.584510] Lustre: DEBUG MARKER: oleg217-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 6223.031584] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid in FULL state after 0 sec [ 6225.126644] LustreError: lustre-OST0000-osc-MDT0001: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6225.133035] Lustre: lustre-OST0000-osc-MDT0001: Connection restored to 0@lo (at 0@lo) [ 6226.495277] Lustre: DEBUG MARKER: oleg217-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff9912d0410000.ost_server_uuid 50 [ 6227.928786] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff9912d0410000.ost_server_uuid in FULL state after 0 sec [ 6233.057061] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 6233.069836] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6233.077144] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6233.079346] Lustre: Skipped 1 previous similar message [ 6235.617166] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6235.622108] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 6237.432215] LustreError: 222892:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 6237.445401] LustreError: 222892:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 6237.565238] Lustre: server umount lustre-OST0000 complete [ 6244.322520] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 6244.335938] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6244.345147] Lustre: Skipped 1 previous similar message [ 6244.351365] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6244.354599] Lustre: Skipped 1 previous similar message [ 6247.295713] LustreError: 223094:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup [ 6247.297618] LustreError: 223094:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 6247.404702] Lustre: server umount lustre-MDT0000 complete [ 6250.840817] LustreError: 219152:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763340565 with bad export cookie 10453591982170719564 [ 6250.856273] LustreError: MGC192.168.202.117@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6250.864647] LustreError: 219152:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 6251.100539] LustreError: 223295:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [ 6251.106587] LustreError: 223295:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 6251.322610] Lustre: server umount lustre-MDT0001 complete [ 6264.517755] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing unload_modules_local [ 6267.944993] Key type lgssc unregistered [ 6268.359852] LNet: 223827:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 6268.385599] LNetError: 223827:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 6268.408886] LNet: Removed LNI 192.168.202.117@tcp [ 6269.450131] Key type .llcrypt unregistered [ 6269.451611] Key type ._llcrypt unregistered [ 6296.738571] Key type ._llcrypt registered [ 6296.741254] Key type .llcrypt registered [ 6296.831940] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing load_modules_local [ 6297.406540] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 6297.510161] alg: No test for adler32 (adler32-zlib) [ 6298.544145] Lustre: Lustre: Build Version: 2.16.61_54_g3c0e0e6 [ 6298.728293] LNet: Added LNI 192.168.202.117@tcp [8/256/0/180] [ 6300.432190] Key type lgssc registered [ 6301.406710] Lustre: Echo OBD driver; http://www.lustre.org/ [ 6310.350691] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 6310.377059] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6311.867175] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6315.311436] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing set_default_debug -1 all [ 6322.151091] LustreError: 225542:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6322.173678] LustreError: 225542:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 6322.523186] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 6322.950747] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 6326.487465] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing set_default_debug -1 all [ 6328.938063] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 6334.581789] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 6335.019939] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 6337.073338] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49699 to 0x280000401:49729) [ 6340.215403] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing set_default_debug -1 all [ 6348.395175] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6348.503873] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 6348.687577] Lustre: lustre-OST0001: new disk, initializing [ 6348.692535] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 6348.791413] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 6353.720731] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing set_default_debug -1 all [ 6353.917149] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 6353.934669] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 6353.979486] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 6363.251501] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 6370.121739] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 6387.580816] LustreError: lustre-MDT0000-osp-MDT0001: operation out_update to node 0@lo failed: rc = -107 [ 6387.584936] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6387.603358] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6389.730683] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6389.736015] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6393.773427] LustreError: 228909:0:(obd_class.h:479:obd_check_dev()) Device 26 not setup [ 6393.834541] Lustre: server umount lustre-MDT0000 complete [ 6394.864555] LustreError: 225542:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6394.903060] LustreError: 225542:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 6396.780832] LustreError: 228687:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763340711 with bad export cookie 1253846483939236201 [ 6396.783909] LustreError: MGC192.168.202.117@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6396.798388] LustreError: 228687:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 6396.930221] LustreError: 229111:0:(obd_class.h:479:obd_check_dev()) Device 25 not setup [ 6396.934382] LustreError: 229111:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 6397.084153] Lustre: server umount lustre-MDT0001 complete [ 6410.887111] LustreError: 229311:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 6410.908794] LustreError: 229311:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 6410.946759] Lustre: server umount lustre-OST0000 complete [ 6424.076452] LustreError: 229515:0:(obd_class.h:479:obd_check_dev()) Device 2 not setup [ 6424.081317] LustreError: 229515:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 6424.230414] Lustre: server umount lustre-OST0001 complete [ 6426.822270] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro [ 6426.877356] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6428.049569] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro [ 6428.152290] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6429.038209] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: ,errors=remount-ro [ 6429.115677] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6430.025945] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: ,errors=remount-ro [ 6430.100843] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6443.302932] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing load_modules_local [ 6453.589529] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6454.121676] LustreError: 230926:0:(ldlm_lib.c:1179:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6454.225256] Lustre: mylustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6458.063586] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing set_default_debug -1 all [ 6459.366057] LustreError: 230926:0:(ldlm_lib.c:1179:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6463.468248] LustreError: 230926:0:(ldlm_lib.c:1179:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6465.682884] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 6466.127555] Lustre: mylustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 6469.425894] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing set_default_debug -1 all [ 6472.229360] Lustre: 232035:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 6478.819340] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 6480.297216] LustreError: 232391:0:(ldlm_lib.c:1179:target_handle_connect()) mylustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6480.329080] Lustre: mylustre-OST0000: new connection from mylustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49699 to 0x280000401:49761) [ 6485.041816] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing set_default_debug -1 all [ 6490.617080] Lustre: mylustre-OST0000: new connection from mylustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:100 to 0x280000400:129) [ 6492.736650] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 6493.076801] Lustre: mylustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 6493.080739] Lustre: Skipped 1 previous similar message [ 6498.357439] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing set_default_debug -1 all [ 6505.711699] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 6509.061296] Lustre: 233882:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 6523.875202] Lustre: mylustre-MDT0000-lwp-MDT0001: Connection to mylustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6523.876727] Lustre: mylustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6523.881393] Lustre: Skipped 4 previous similar messages [ 6523.888105] Lustre: Skipped 6 previous similar messages [ 6528.997057] Lustre: mylustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6529.004548] Lustre: Skipped 3 previous similar messages [ 6534.115355] Lustre: mylustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6534.130560] Lustre: Skipped 3 previous similar messages [ 6537.189077] Lustre: mylustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 6537.309667] LustreError: 234191:0:(obd_class.h:479:obd_check_dev()) Device 10 not setup [ 6537.340796] LustreError: 234191:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 6537.432328] Lustre: server umount mylustre-MDT0000 complete [ 6539.237516] LustreError: 230925:0:(ldlm_lib.c:1179:target_handle_connect()) mylustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6539.242977] LustreError: 230925:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 6 previous similar messages [ 6540.958847] LustreError: 230902:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763340855 with bad export cookie 1253846483939239064 [ 6540.964675] LustreError: MGC192.168.202.117@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6541.335173] Lustre: server umount mylustre-MDT0001 complete [ 6555.365993] LustreError: 234596:0:(obd_class.h:479:obd_check_dev()) Device 23 not setup [ 6555.389725] LustreError: 234596:0:(obd_class.h:479:obd_check_dev()) Skipped 15 previous similar messages [ 6555.541101] Lustre: server umount mylustre-OST0000 complete [ 6568.825457] Lustre: server umount mylustre-OST0001 complete [ 6572.179214] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro [ 6572.278346] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6573.606636] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro [ 6573.748474] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6574.813659] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: ,errors=remount-ro [ 6574.904987] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6575.908642] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: ,errors=remount-ro [ 6575.995600] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6589.775684] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing load_modules_local [ 6597.892421] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6598.232406] LustreError: 236209:0:(ldlm_lib.c:1179:target_handle_connect()) tfs-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6598.335566] Lustre: tfs-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6601.562353] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing set_default_debug -1 all [ 6610.744853] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 6615.445452] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing set_default_debug -1 all [ 6618.844979] Lustre: 237321:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 6624.717842] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 6630.152882] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing set_default_debug -1 all [ 6636.344403] Lustre: tfs-OST0000: new connection from tfs-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:100 to 0x280000400:161) [ 6638.526482] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 6640.610330] Lustre: tfs-OST0001: new connection from tfs-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:35 to 0x2c0000400:65) [ 6642.676850] Lustre: tfs-OST0000: new connection from tfs-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49699 to 0x280000401:49793) [ 6643.995564] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing set_default_debug -1 all [ 6650.548606] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 6653.440545] Lustre: 239167:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 6667.749024] LustreError: tfs-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 6667.767222] Lustre: tfs-MDT0000-osp-MDT0001: Connection to tfs-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6667.782160] Lustre: Skipped 1 previous similar message [ 6667.793438] Lustre: tfs-MDT0000: Not available for connect from 0@lo (stopping) [ 6670.999788] LustreError: 239416:0:(obd_class.h:479:obd_check_dev()) Device 10 not setup [ 6671.007304] LustreError: 239416:0:(obd_class.h:479:obd_check_dev()) Skipped 9 previous similar messages [ 6671.098757] Lustre: server umount tfs-MDT0000 complete [ 6673.393605] LustreError: 236209:0:(ldlm_lib.c:1179:target_handle_connect()) tfs-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6673.423054] LustreError: 236209:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 6 previous similar messages [ 6674.157502] LustreError: 236184:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763340988 with bad export cookie 1253846483939241017 [ 6674.166827] LustreError: MGC192.168.202.117@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6674.181783] LustreError: 236184:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 6674.566626] Lustre: server umount tfs-MDT0001 complete [ 6687.830662] Lustre: server umount tfs-OST0000 complete [ 6700.770719] Lustre: server umount tfs-OST0001 complete [ 6703.185521] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro [ 6703.276664] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6704.023551] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro [ 6704.090363] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6704.857192] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: ,errors=remount-ro [ 6704.913429] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6705.659679] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: ,errors=remount-ro [ 6705.721986] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6717.265677] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing load_modules_local [ 6726.795980] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6727.483920] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6727.489935] Lustre: Skipped 3 previous similar messages [ 6731.082565] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing set_default_debug -1 all [ 6738.692927] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 6742.469620] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing set_default_debug -1 all [ 6745.430282] Lustre: 242541:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 6751.800628] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 6753.952126] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49699 to 0x280000401:49825) [ 6758.126068] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing set_default_debug -1 all [ 6763.528499] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:100 to 0x280000400:193) [ 6765.497733] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 6767.322254] Lustre: lustre-OST0001: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:35 to 0x2c0000400:97) [ 6770.349307] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing set_default_debug -1 all [ 6777.596762] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 6780.766899] Lustre: 244390:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 6791.582783] Lustre: DEBUG MARKER: == conf-sanity test 104a: Make sure user defined options are reflected in mount ========================================================== 19:58:25 (1763341105) [ 6807.013048] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6807.024516] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6807.037326] Lustre: Skipped 5 previous similar messages [ 6807.062963] Lustre: Skipped 7 previous similar messages [ 6812.413293] LustreError: 244759:0:(obd_class.h:479:obd_check_dev()) Device 10 not setup [ 6812.423134] LustreError: 244759:0:(obd_class.h:479:obd_check_dev()) Skipped 25 previous similar messages [ 6812.498122] Lustre: server umount lustre-MDT0000 complete [ 6815.718046] LustreError: 242544:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763341130 with bad export cookie 1253846483939242928 [ 6815.722245] LustreError: MGC192.168.202.117@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6815.731779] LustreError: 242544:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 6817.250230] LustreError: 241428:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6817.251439] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6817.266683] LustreError: 241428:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 9 previous similar messages [ 6817.277536] Lustre: Skipped 1 previous similar message [ 6830.048297] Lustre: lustre-MDT0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 6830.285757] Lustre: server umount lustre-MDT0001 complete [ 6840.156628] Lustre: server umount lustre-OST0000 complete [ 6849.382418] Lustre: server umount lustre-OST0001 complete [ 6854.026990] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing set_hostid [ 6861.690969] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing load_modules_local [ 6869.020779] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6874.971559] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6880.577363] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6885.842738] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6892.408192] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6892.485330] LDISKFS-fs (dm-0): Mount option "noacl" will be removed by 3.5 [ 6892.485330] Contact linux-ldiskfs@vger.kernel.org if you think we should keep it. [ 6892.485330] [ 6892.512986] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: acl,user_xattr,errors=remount-ro,noacl,no_mbcache,nodelalloc [ 6892.721510] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 6892.752902] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 6892.815841] Lustre: lustre-MDT0000: new disk, initializing [ 6892.900865] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6892.905660] Lustre: Skipped 3 previous similar messages [ 6892.925934] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 6896.614369] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing set_default_debug -1 all [ 6906.173148] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6906.222141] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 6906.295066] Lustre: 247550:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 6906.394266] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 6906.398246] Lustre: Skipped 1 previous similar message [ 6906.471564] Lustre: lustre-MDT0001: new disk, initializing [ 6906.616697] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 6906.623210] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 6910.482673] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing set_default_debug -1 all [ 6920.971555] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6921.064852] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 6921.395289] Lustre: lustre-OST0000: new disk, initializing [ 6921.398202] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 6923.131778] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 6923.137641] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 6923.236889] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 6927.072292] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing set_default_debug -1 all [ 6935.257948] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6935.309248] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 6935.424711] Lustre: lustre-OST0001: new disk, initializing [ 6935.427621] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 6936.890398] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 6936.900989] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 6937.013078] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 6939.929524] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing set_default_debug -1 all [ 6949.634584] Lustre: DEBUG MARKER: == conf-sanity test 104b: Mount uses last flock argument ========================================================== 20:01:03 (1763341263) [ 6960.947440] Lustre: DEBUG MARKER: == conf-sanity test 105: check file creation for ro and rw bind mnt pt ========================================================== 20:01:14 (1763341274) [ 6974.437637] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 6974.447249] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6974.468024] Lustre: Skipped 1 previous similar message [ 6974.477133] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6974.479818] Lustre: Skipped 12 previous similar messages [ 6980.480266] LustreError: 250987:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 6980.492252] LustreError: 250987:0:(obd_class.h:479:obd_check_dev()) Skipped 25 previous similar messages [ 6980.539215] Lustre: server umount lustre-OST0000 complete [ 6990.471234] Lustre: server umount lustre-MDT0000 complete [ 6994.040854] LustreError: 247543:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763341308 with bad export cookie 1253846483939244713 [ 6994.044602] LustreError: MGC192.168.202.117@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6994.087679] LustreError: 247543:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 7000.547618] Lustre: server umount lustre-MDT0001 complete [ 7011.484090] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing unload_modules_local [ 7027.449406] Lustre: server umount lustre-OST0001 complete [ 7032.993568] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing set_hostid [ 7046.488960] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing load_modules_local [ 7054.911895] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 7060.787751] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 7066.653852] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 7072.580166] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 7079.229292] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 7079.311820] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7079.520628] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 7079.562301] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 7079.621628] Lustre: lustre-MDT0000: new disk, initializing [ 7079.739546] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 7083.401425] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing set_default_debug -1 all [ 7094.197526] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 7094.253604] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 7094.346549] Lustre: 254473:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 7094.674420] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 7098.938976] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing set_default_debug -1 all [ 7106.627528] Lustre: DEBUG MARKER: oleg217-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 7110.260851] Lustre: DEBUG MARKER: oleg217-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 7116.934532] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 7117.000967] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 7117.193635] Lustre: lustre-OST0000: new disk, initializing [ 7117.195595] Lustre: Skipped 1 previous similar message [ 7117.199548] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 7117.201510] Lustre: Skipped 2 previous similar messages [ 7119.196756] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 7119.200550] Lustre: Skipped 1 previous similar message [ 7119.231385] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 7119.376859] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 7123.273559] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing set_default_debug -1 all [ 7131.534739] Lustre: DEBUG MARKER: oleg217-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 7139.810393] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 7139.813758] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7139.822720] LustreError: Skipped 2 previous similar messages [ 7139.826688] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7139.829462] Lustre: Skipped 6 previous similar messages [ 7139.839817] Lustre: Skipped 8 previous similar messages [ 7142.624248] LustreError: 256916:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 7142.634422] LustreError: 256916:0:(obd_class.h:479:obd_check_dev()) Skipped 25 previous similar messages [ 7142.800349] Lustre: server umount lustre-OST0000 complete [ 7152.902743] Lustre: server umount lustre-MDT0000 complete [ 7155.174746] LustreError: 254484:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7155.180816] LustreError: 254484:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 15 previous similar messages [ 7156.419367] LustreError: 254466:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763341471 with bad export cookie 1253846483939246183 [ 7156.420624] LustreError: MGC192.168.202.117@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7156.433524] LustreError: 254466:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 7156.806313] Lustre: server umount lustre-MDT0001 complete [ 7166.648526] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing unload_modules_local [ 7169.377725] Key type lgssc unregistered [ 7169.684602] LNet: 257851:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 7169.698109] LNetError: 257851:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 7169.711419] LNet: Removed LNI 192.168.202.117@tcp [ 7170.659207] Key type .llcrypt unregistered [ 7170.660665] Key type ._llcrypt unregistered [ 7193.772197] Key type ._llcrypt registered [ 7193.773723] Key type .llcrypt registered [ 7193.880971] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing load_modules_local [ 7194.640204] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 7194.649871] alg: No test for adler32 (adler32-zlib) [ 7195.699560] Lustre: Lustre: Build Version: 2.16.61_54_g3c0e0e6 [ 7195.914290] LNet: Added LNI 192.168.202.117@tcp [8/256/0/180] [ 7197.593331] Key type lgssc registered [ 7198.585984] Lustre: Echo OBD driver; http://www.lustre.org/ [ 7204.964417] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 7204.992434] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7206.441956] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7210.131457] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing set_default_debug -1 all [ 7215.590516] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 7215.596922] LustreError: 259295:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7215.607188] LustreError: 259295:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 7215.847194] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7218.461764] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing set_default_debug -1 all [ 7223.179237] Lustre: DEBUG MARKER: oleg217-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 7226.340181] Lustre: DEBUG MARKER: oleg217-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 7232.396747] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 7232.683412] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7237.310554] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing set_default_debug -1 all [ 7241.648544] Lustre: DEBUG MARKER: oleg217-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 7242.489703] Lustre: Setting parameter lustre-OST0000.ost.unknown_param=50 in log lustre-OST0000 [ 7246.305615] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 7246.312159] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7246.322122] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7249.991171] LustreError: 261587:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 7250.107325] Lustre: server umount lustre-OST0000 complete [ 7255.523312] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7255.528946] Lustre: Skipped 1 previous similar message [ 7255.529226] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7255.544855] Lustre: Skipped 1 previous similar message [ 7260.642385] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7260.644834] Lustre: Skipped 1 previous similar message [ 7265.762156] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7265.766901] Lustre: Skipped 1 previous similar message [ 7267.298189] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 7267.379943] LustreError: 261792:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup [ 7267.384416] LustreError: 261792:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 7267.449361] Lustre: server umount lustre-MDT0000 complete [ 7270.882580] LustreError: 259291:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7270.902641] LustreError: 259291:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 7271.052721] LustreError: 259275:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763341585 with bad export cookie 1049397892994257381 [ 7271.058188] LustreError: MGC192.168.202.117@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7271.066685] LustreError: 259275:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 7271.224357] LustreError: 261995:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [ 7271.239564] LustreError: 261995:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 7271.527427] Lustre: server umount lustre-MDT0001 complete [ 7281.561234] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing unload_modules_local [ 7284.215128] Key type lgssc unregistered [ 7284.531829] LNet: 262529:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 7284.540314] LNetError: 262529:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 7284.555996] LNet: Removed LNI 192.168.202.117@tcp [ 7285.230545] Key type .llcrypt unregistered [ 7285.231869] Key type ._llcrypt unregistered [ 7301.736632] Key type ._llcrypt registered [ 7301.747662] Key type .llcrypt registered [ 7301.886196] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing load_modules_local [ 7302.923835] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 7302.956984] alg: No test for adler32 (adler32-zlib) [ 7304.158558] Lustre: Lustre: Build Version: 2.16.61_54_g3c0e0e6 [ 7304.347843] LNet: Added LNI 192.168.202.117@tcp [8/256/0/180] [ 7305.976290] Key type lgssc registered [ 7307.273053] Lustre: Echo OBD driver; http://www.lustre.org/ [ 7315.591324] Lustre: lustre-OST0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 7315.612154] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 7331.168588] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 7336.295990] LustreError: 263703:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.202.117@tcp: failed processing log, type 4: rc = -110 [ 7363.040381] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7368.436510] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing set_default_debug -1 all [ 7374.471403] Lustre: DEBUG MARKER: oleg217-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 7376.623225] LustreError: 264368:0:(obd_class.h:479:obd_check_dev()) Device 1 not setup [ 7376.786516] Lustre: server umount lustre-OST0000 complete [ 7392.626960] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing unload_modules_local [ 7395.603881] Key type lgssc unregistered [ 7395.884974] LNet: 265204:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 7395.890844] LNetError: 265204:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 7395.913121] LNet: Removed LNI 192.168.202.117@tcp [ 7396.781473] Key type .llcrypt unregistered [ 7396.792058] Key type ._llcrypt unregistered [ 7429.777433] Key type ._llcrypt registered [ 7429.779052] Key type .llcrypt registered [ 7429.879694] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing load_modules_local [ 7430.866262] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 7430.934940] alg: No test for adler32 (adler32-zlib) [ 7431.979837] Lustre: Lustre: Build Version: 2.16.61_54_g3c0e0e6 [ 7432.193759] LNet: Added LNI 192.168.202.117@tcp [8/256/0/180] [ 7433.904560] Key type lgssc registered [ 7434.860716] Lustre: Echo OBD driver; http://www.lustre.org/ [ 7445.079057] loop0: detected capacity change from 0 to 204800000 [ 7445.293577] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 7445.301592] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 7445.495899] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 7445.542804] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 7446.425697] loop0: detected capacity change from 0 to 204800000 [ 7446.574031] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 7446.581379] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 7446.838356] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 7446.896147] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 7447.576340] loop0: detected capacity change from 0 to 204800000 [ 7447.704591] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 7447.709692] blk_update_request: operation not supported error, dev loop0, sector 8224 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 7447.796596] blk_update_request: operation not supported error, dev loop0, sector 58264 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 7447.882333] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 7448.489289] loop0: detected capacity change from 0 to 204800000 [ 7448.581886] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 7448.742582] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 7449.335797] loop0: detected capacity change from 0 to 204800000 [ 7449.356899] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 7452.456603] loop0: detected capacity change from 0 to 204800000 [ 7452.474434] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 7455.451613] loop0: detected capacity change from 0 to 204800000 [ 7455.479994] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 7458.595726] loop0: detected capacity change from 0 to 204800000 [ 7458.633562] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 7461.685893] loop0: detected capacity change from 0 to 204800000 [ 7461.759827] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 7461.813239] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 7461.824947] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7461.874562] Lustre: 268607:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 7461.882436] Lustre: 268607:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x10000-MDT0000' with [0x200000005:0x1:0x0]: rc = 0 [ 7461.888618] Lustre: 268607:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x1010000-MDT0000' with [0x200000005:0x2:0x0]: rc = 0 [ 7461.894959] Lustre: 268607:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000003:0xb:0x0]: rc = 0 [ 7461.901368] Lustre: 268607:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000003:0xc:0x0]: rc = 0 [ 7461.908236] Lustre: 268607:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000003:0xa:0x0]: rc = 0 [ 7461.913263] Lustre: 268607:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x2010000-MDT0000' with [0x200000005:0x3:0x0]: rc = 0 [ 7461.919393] Lustre: 268607:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index 'nodemap' with [0x200000003:0x3:0x0]: rc = 0 [ 7461.931833] Lustre: 268607:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x2020000' with [0x200000006:0x2020000:0x0]: rc = 0 [ 7461.937604] Lustre: 268607:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x20000' with [0x200000006:0x20000:0x0]: rc = 0 [ 7461.944504] Lustre: 268607:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x1020000' with [0x200000006:0x1020000:0x0]: rc = 0 [ 7461.953753] Lustre: 268607:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000006:0x1010000:0x0]: rc = 0 [ 7461.960993] Lustre: 268607:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000006:0x2010000:0x0]: rc = 0 [ 7461.968693] Lustre: 268607:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000006:0x10000:0x0]: rc = 0 [ 7466.508300] LustreError: 268875:0:(obd_class.h:479:obd_check_dev()) Device 2 not setup [ 7466.632760] Lustre: server umount MGS complete [ 7467.274712] loop0: detected capacity change from 0 to 204800000 [ 7467.400704] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7467.736063] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7467.777582] Lustre: lustre-MDT0000: Aborting client recovery [ 7468.737869] loop1: detected capacity change from 0 to 204800000 [ 7468.911142] LDISKFS-fs (loop1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 7468.988631] LDISKFS-fs (loop1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 7469.029868] Lustre: 269045:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 7469.036548] Lustre: 269045:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0001: restore index '0x10000-MDT0001' with [0x200000005:0x1:0x0]: rc = 0 [ 7469.048699] Lustre: 269045:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0001: restore index '0x2010000' with [0x200000003:0x6:0x0]: rc = 0 [ 7469.056286] Lustre: 269045:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0001: restore index '0x1010000' with [0x200000003:0x5:0x0]: rc = 0 [ 7469.064984] Lustre: 269045:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0001: restore index '0x1010000-MDT0001' with [0x200000005:0x2:0x0]: rc = 0 [ 7469.075424] Lustre: 269045:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0001: restore index '0x2010000-MDT0001' with [0x200000005:0x3:0x0]: rc = 0 [ 7469.082979] Lustre: 269045:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0001: restore index '0x10000' with [0x200000003:0x4:0x0]: rc = 0 [ 7469.100890] Lustre: Found index 1 for lustre-MDT0001, updating log [ 7469.324144] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7469.384499] Lustre: lustre-MDT0001: Aborting client recovery [ 7470.221267] loop2: detected capacity change from 0 to 204800000 [ 7470.291221] LDISKFS-fs (loop2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 7470.319728] LDISKFS-fs (loop2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 7470.366690] Lustre: 269127:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 7470.373663] Lustre: 269127:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0000: restore index '0x1020000-OST0000' with [0x200000005:0x2:0x0]: rc = 0 [ 7470.382361] Lustre: 269127:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0000: restore index '0x2020000-OST0000' with [0x200000005:0x3:0x0]: rc = 0 [ 7470.390371] Lustre: 269127:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0000: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 7470.398473] Lustre: 269127:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0000: restore index '0x20000-OST0000' with [0x200000005:0x1:0x0]: rc = 0 [ 7470.406830] Lustre: 269127:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0000: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 7470.417945] Lustre: 269127:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0000: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 7470.516228] Lustre: Found index 0 for lustre-OST0000, updating log [ 7470.579101] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7470.594935] Lustre: lustre-OST0000: Aborting recovery [ 7471.147506] loop3: detected capacity change from 0 to 204800000 [ 7471.271172] LDISKFS-fs (loop3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 7471.316927] LDISKFS-fs (loop3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 7471.340964] Lustre: 269214:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 7471.348475] Lustre: 269214:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0001: restore index '0x20000-OST0001' with [0x200000005:0x1:0x0]: rc = 0 [ 7471.353920] Lustre: 269214:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0001: restore index '0x1020000-OST0001' with [0x200000005:0x2:0x0]: rc = 0 [ 7471.361389] Lustre: 269214:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0001: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 7471.368675] Lustre: 269214:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0001: restore index '0x2020000-OST0001' with [0x200000005:0x3:0x0]: rc = 0 [ 7471.377579] Lustre: 269214:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0001: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 7471.387922] Lustre: 269214:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0001: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 7471.506896] Lustre: lustre-OST0001: Aborting recovery [ 7476.744655] LustreError: 268987:0:(lod_dev.c:510:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 9, retries 0, failed: rc = -108 [ 7477.748903] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 7477.762981] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 7477.776509] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000bd0 [ 7477.833150] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000bd0 [ 7478.761105] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:5 to 0x280000400:33) [ 7478.763112] Lustre: lustre-OST0001: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:5 to 0x2c0000400:33) [ 7484.087612] LustreError: 268955:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0000: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0000 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0000/identity_upcall, time 1681us: rc = -2 [ 7484.589335] LustreError: 269464:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0000: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0000 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0000/identity_upcall, time 534us: rc = -2 [ 7484.599541] LustreError: 269464:0:(mdt_identity.c:115:mdt_identity_do_upcall()) Skipped 30 previous similar messages [ 7499.237178] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7499.240943] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7499.249937] Lustre: Skipped 3 previous similar messages [ 7499.257019] Lustre: Skipped 3 previous similar messages [ 7502.585021] LustreError: 269567:0:(obd_class.h:479:obd_check_dev()) Device 10 not setup [ 7502.587460] LustreError: 269567:0:(obd_class.h:479:obd_check_dev()) Skipped 1 previous similar message [ 7502.761745] Lustre: server umount lustre-MDT0000 complete [ 7503.441637] LustreError: 269568:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763341818 with bad export cookie 7115055088380199438 [ 7503.443084] LustreError: MGC192.168.202.117@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7503.459272] LustreError: 269568:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 7504.356036] LustreError: 268953:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7504.363368] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7504.363646] LustreError: 268953:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 7504.368874] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 7504.369096] Lustre: Skipped 1 previous similar message [ 7504.375208] Lustre: Skipped 1 previous similar message [ 7508.385144] LustreError: 268951:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7508.388599] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 7508.398761] LustreError: 268951:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 7508.423355] Lustre: Skipped 1 previous similar message [ 7509.616553] LustreError: 269619:0:(obd_class.h:479:obd_check_dev()) Device 18 not setup [ 7509.620902] LustreError: 269619:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 7509.821207] Lustre: server umount lustre-MDT0001 complete [ 7516.677604] LustreError: 269671:0:(obd_class.h:479:obd_check_dev()) Device 23 not setup [ 7516.689702] LustreError: 269671:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 7516.780212] Lustre: server umount lustre-OST0000 complete [ 7523.572769] LustreError: 269722:0:(obd_class.h:479:obd_check_dev()) Device 2 not setup [ 7523.577150] LustreError: 269722:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 7523.743822] Lustre: server umount lustre-OST0001 complete [ 7531.073497] Lustre: DEBUG MARKER: == conf-sanity test 109a: test lctl clear_conf fsname ==== 20:10:44 (1763341844) [ 7553.913458] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing set_hostid [ 7562.069148] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing load_modules_local [ 7570.062283] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 7575.578282] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 7580.962160] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 7586.637862] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 7592.961674] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 7593.034545] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7593.201564] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 7593.230175] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 7593.285876] Lustre: lustre-MDT0000: new disk, initializing [ 7593.353341] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7593.362256] Lustre: Skipped 1 previous similar message [ 7593.400634] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 7596.990222] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing set_default_debug -1 all [ 7606.184992] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 7606.274884] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 7606.457474] Lustre: 273272:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 7606.574952] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 7606.595593] Lustre: Skipped 1 previous similar message [ 7606.731428] Lustre: lustre-MDT0001: new disk, initializing [ 7606.847572] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7606.895675] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 7606.914529] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 7610.881595] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing set_default_debug -1 all [ 7617.473312] Lustre: DEBUG MARKER: oleg217-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 7620.657078] Lustre: DEBUG MARKER: oleg217-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 7627.062825] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 7627.136812] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 7627.323799] Lustre: lustre-OST0000: new disk, initializing [ 7627.328473] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 7627.416060] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7632.248863] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing set_default_debug -1 all [ 7633.489238] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 7633.492481] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 7633.600589] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 7639.259639] Lustre: DEBUG MARKER: oleg217-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 7644.251476] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [ 7657.309749] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [ 7663.635335] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [ 7700.449698] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7700.452048] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 7700.463308] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7700.469609] LustreError: Skipped 1 previous similar message [ 7705.576023] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7705.593128] Lustre: Skipped 1 previous similar message [ 7706.506104] LustreError: 276831:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 7706.508989] LustreError: 276831:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 7706.625518] Lustre: server umount lustre-OST0000 complete [ 7710.695198] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7710.714309] Lustre: Skipped 2 previous similar messages [ 7715.809358] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7715.821489] Lustre: Skipped 4 previous similar messages [ 7715.901733] Lustre: server umount lustre-MDT0000 complete [ 7719.489830] LustreError: 273263:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763342034 with bad export cookie 7115055088380202147 [ 7719.495071] LustreError: MGC192.168.202.117@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7719.499500] LustreError: 273263:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 7719.834703] Lustre: server umount lustre-MDT0001 complete [ 7728.021815] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7731.689195] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing set_default_debug -1 all [ 7736.612787] LustreError: 278177:0:(obd_class.h:479:obd_check_dev()) Device 2 not setup [ 7736.622556] LustreError: 278177:0:(obd_class.h:479:obd_check_dev()) Skipped 17 previous similar messages [ 7736.699873] Lustre: server umount MGS complete [ 7747.205242] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7747.598820] LustreError: 278801:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7747.618415] LustreError: 278813:0:(qmt_pool.c:1523:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [ 7747.669478] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7750.900643] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing set_default_debug -1 all [ 7752.679601] LustreError: 278801:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7757.798449] LustreError: 278801:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7758.756993] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 7762.796815] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing set_default_debug -1 all [ 7767.800462] Lustre: DEBUG MARKER: oleg217-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 7770.872426] Lustre: DEBUG MARKER: oleg217-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 7776.752401] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 7782.287917] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing set_default_debug -1 all [ 7787.318615] Lustre: DEBUG MARKER: oleg217-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 7818.212644] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 7818.222141] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7818.243847] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7818.251425] Lustre: Skipped 1 previous similar message [ 7818.268317] Lustre: Skipped 1 previous similar message [ 7819.125175] LustreError: 281462:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 7819.140184] LustreError: 281462:0:(obd_class.h:479:obd_check_dev()) Skipped 1 previous similar message [ 7819.251315] Lustre: server umount lustre-OST0000 complete [ 7837.152144] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 7837.367972] Lustre: server umount lustre-MDT0000 complete [ 7837.668738] LustreError: 278800:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7837.680334] LustreError: 278800:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 7840.769695] LustreError: 281199:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763342155 with bad export cookie 7115055088380203974 [ 7840.770864] LustreError: MGC192.168.202.117@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7840.792329] LustreError: 281199:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 7841.251735] Lustre: server umount lustre-MDT0001 complete [ 7851.269779] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing unload_modules_local [ 7854.311727] Key type lgssc unregistered [ 7854.593996] LNet: 282399:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 7854.603866] LNetError: 282399:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 7854.637094] LNet: Removed LNI 192.168.202.117@tcp [ 7855.428191] Key type .llcrypt unregistered [ 7855.429889] Key type ._llcrypt unregistered [ 7889.285372] Key type ._llcrypt registered [ 7889.286961] Key type .llcrypt registered [ 7889.411838] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing set_hostid [ 7902.994989] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing load_modules_local [ 7903.986803] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 7904.057808] alg: No test for adler32 (adler32-zlib) [ 7905.146978] Lustre: Lustre: Build Version: 2.16.61_54_g3c0e0e6 [ 7905.354962] LNet: Added LNI 192.168.202.117@tcp [8/256/0/180] [ 7907.016168] Key type lgssc registered [ 7907.960352] Lustre: Echo OBD driver; http://www.lustre.org/ [ 7915.488310] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 7920.993802] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 7927.167448] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 7932.252330] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 7938.518179] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 7938.576600] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 7938.591597] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7939.868820] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 7939.911412] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 7940.003951] Lustre: lustre-MDT0000: new disk, initializing [ 7940.103713] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7940.118304] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 7943.842242] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing set_default_debug -1 all [ 7953.235087] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 7953.318235] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 7953.412112] Lustre: 286332:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 7953.491880] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 7953.499452] Lustre: Skipped 1 previous similar message [ 7953.575635] Lustre: lustre-MDT0001: new disk, initializing [ 7953.682815] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7953.728117] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 7953.736025] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 7957.500769] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing set_default_debug -1 all [ 7964.452792] Lustre: DEBUG MARKER: oleg217-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 7967.719137] Lustre: DEBUG MARKER: oleg217-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 7973.463239] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 7973.527581] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 7973.704666] Lustre: lustre-OST0000: new disk, initializing [ 7973.707636] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 7973.807492] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7979.119723] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing set_default_debug -1 all [ 7980.095151] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 7980.107617] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 7980.303048] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 7986.646374] Lustre: DEBUG MARKER: oleg217-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 7994.455283] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [ 8005.826272] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [ 8009.721049] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [ 8014.512190] Lustre: Modifying parameter lustre-client.llite.max_read_ahead_mb=64 in log lustre-client [ 8062.434899] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 8062.437937] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8062.445113] LustreError: Skipped 1 previous similar message [ 8062.461227] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8062.466932] Lustre: Skipped 1 previous similar message [ 8067.556897] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8072.674556] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8072.683704] Lustre: Skipped 2 previous similar messages [ 8074.208269] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 8074.242109] LustreError: 289804:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 8074.323651] Lustre: server umount lustre-OST0000 complete [ 8077.794984] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8077.795773] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8077.801084] Lustre: Skipped 2 previous similar messages [ 8082.917565] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8082.923623] Lustre: Skipped 2 previous similar messages [ 8083.893673] LustreError: 290006:0:(obd_class.h:479:obd_check_dev()) Device 22 not setup [ 8083.900554] LustreError: 290006:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 8084.010341] Lustre: server umount lustre-MDT0000 complete [ 8087.842477] LustreError: 288080:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763342402 with bad export cookie 17787509403601344900 [ 8087.847680] LustreError: MGC192.168.202.117@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8087.863767] LustreError: 288080:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 8087.981156] LustreError: 290207:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 8087.984224] LustreError: 290207:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 8088.115740] Lustre: server umount lustre-MDT0001 complete [ 8096.833615] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8100.275274] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing set_default_debug -1 all [ 8105.740341] LustreError: 291152:0:(obd_class.h:479:obd_check_dev()) Device 2 not setup [ 8105.743067] LustreError: 291152:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 8105.837160] Lustre: server umount MGS complete [ 8117.649081] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8118.136642] LustreError: 291789:0:(qmt_pool.c:1523:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [ 8118.230438] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8123.104950] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing set_default_debug -1 all [ 8123.362255] LustreError: 291776:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8123.368291] LustreError: 291776:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 8127.462398] LustreError: 291776:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8132.112765] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 8132.454311] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8136.495770] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing set_default_debug -1 all [ 8141.782503] Lustre: DEBUG MARKER: oleg217-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 8145.457218] Lustre: DEBUG MARKER: oleg217-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 8152.064756] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 8152.396621] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8158.483766] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing set_default_debug -1 all [ 8164.146505] Lustre: DEBUG MARKER: oleg217-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 8197.601309] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 8197.604805] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8197.642099] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8198.627912] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 8208.352294] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 8208.389497] LustreError: 294432:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 8208.396498] LustreError: 294432:0:(obd_class.h:479:obd_check_dev()) Skipped 1 previous similar message [ 8208.469337] Lustre: server umount lustre-OST0000 complete [ 8213.986058] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8213.991077] Lustre: Skipped 1 previous similar message [ 8213.994609] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8213.997032] Lustre: Skipped 4 previous similar messages [ 8218.461598] LustreError: 294635:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup [ 8218.476097] LustreError: 294635:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 8218.597641] Lustre: server umount lustre-MDT0000 complete [ 8219.108678] LustreError: 291776:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8219.124032] LustreError: 291776:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 8221.929974] LustreError: 291756:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763342536 with bad export cookie 17787509403601346727 [ 8221.932087] LustreError: MGC192.168.202.117@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8221.947481] LustreError: 291756:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 8222.311964] Lustre: server umount lustre-MDT0001 complete [ 8233.363963] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing unload_modules_local [ 8236.294231] Key type lgssc unregistered [ 8236.697314] LNet: 295368:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8236.702557] LNetError: 295368:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8236.716951] LNet: Removed LNI 192.168.202.117@tcp [ 8237.708655] Key type .llcrypt unregistered [ 8237.710197] Key type ._llcrypt unregistered [ 8257.157641] Key type ._llcrypt registered [ 8257.159355] Key type .llcrypt registered [ 8257.249894] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing set_hostid [ 8269.156610] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing load_modules_local [ 8270.066197] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8270.088964] alg: No test for adler32 (adler32-zlib) [ 8271.226327] Lustre: Lustre: Build Version: 2.16.61_54_g3c0e0e6 [ 8271.497622] LNet: Added LNI 192.168.202.117@tcp [8/256/0/180] [ 8273.195139] Key type lgssc registered [ 8274.606529] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8283.335375] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 8289.128614] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 8297.349957] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing load_modules_local [ 8305.417211] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 8311.899126] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 8311.955902] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 8311.967500] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8313.299728] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 8313.342119] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 8313.410839] Lustre: lustre-MDT0000: new disk, initializing [ 8313.526931] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8313.552906] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 8317.244150] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing set_default_debug -1 all [ 8327.693602] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 8327.778520] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 8327.981944] Lustre: lustre-OST0000: new disk, initializing [ 8327.990940] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 8328.004102] Lustre: Skipped 1 previous similar message [ 8328.091275] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8328.250154] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 8328.261599] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 8328.300982] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 8334.914952] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing set_default_debug -1 all [ 9104.354053] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9104.372780] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9107.480162] LustreError: 300380:0:(obd_class.h:479:obd_check_dev()) Device 13 not setup [ 9107.653435] Lustre: server umount lustre-MDT0000 complete [ 9110.707958] LustreError: 298538:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763343425 with bad export cookie 12128989625222235153 [ 9110.714812] LustreError: MGC192.168.202.117@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9110.769682] LustreError: 300581:0:(obd_class.h:479:obd_check_dev()) Device 2 not setup [ 9110.773464] LustreError: 300581:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 9110.921127] Lustre: server umount lustre-OST0000 complete [ 9132.250122] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing set_hostid [ 9140.548720] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing load_modules_local [ 9147.959788] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 9153.396979] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 9158.710851] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 9163.855679] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 9169.517524] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 9169.577356] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9169.807367] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 9169.843136] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 9169.925741] Lustre: lustre-MDT0000: new disk, initializing [ 9170.035738] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9170.061294] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 9173.429778] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing set_default_debug -1 all [ 9181.808764] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 9181.871702] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 9181.946395] Lustre: 303626:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 9182.008266] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 9182.013727] Lustre: Skipped 1 previous similar message [ 9182.088682] Lustre: lustre-MDT0001: new disk, initializing [ 9182.160706] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9182.183333] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 9182.193695] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 9184.954216] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing set_default_debug -1 all [ 9190.796368] Lustre: DEBUG MARKER: oleg217-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 9193.796121] Lustre: DEBUG MARKER: oleg217-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 9198.766857] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 9198.838719] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 9199.047728] Lustre: lustre-OST0000: new disk, initializing [ 9199.054715] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 9199.123129] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9202.866231] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing set_default_debug -1 all [ 9205.798189] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 9205.804402] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 9205.888942] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 9208.694376] Lustre: DEBUG MARKER: oleg217-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 9212.110584] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 9212.297665] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 9215.104521] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 9215.285336] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 9221.092836] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 9221.096053] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9221.101273] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9221.126373] Lustre: Skipped 1 previous similar message [ 9222.788531] LustreError: 306561:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 9222.791777] LustreError: 306561:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 9222.939911] Lustre: server umount lustre-OST0000 complete [ 9226.208937] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9226.216179] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9226.221100] Lustre: Skipped 1 previous similar message [ 9226.227218] Lustre: Skipped 1 previous similar message [ 9231.329604] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9231.340853] Lustre: Skipped 2 previous similar messages [ 9232.369177] LustreError: 306762:0:(obd_class.h:479:obd_check_dev()) Device 22 not setup [ 9232.373179] LustreError: 306762:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 9232.754988] Lustre: server umount lustre-MDT0000 complete [ 9236.176959] LustreError: 303616:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763343550 with bad export cookie 12128989625225176000 [ 9236.189908] LustreError: MGC192.168.202.117@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9236.196233] LustreError: 303616:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 9236.479959] LustreError: 306965:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 9236.485207] LustreError: 306965:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 9236.662919] Lustre: server umount lustre-MDT0001 complete [ 9240.605252] Lustre: DEBUG MARKER: SKIP: conf-sanity test_111 skipping SLOW test 111 [ 9242.267867] Lustre: DEBUG MARKER: == conf-sanity test 112a: mount OST with no_create option ========================================================== 20:39:15 (1763343555) [ 9248.233501] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9248.623740] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9251.718501] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing set_default_debug -1 all [ 9253.858525] LustreError: 307537:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9253.868562] LustreError: 307537:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 9258.982673] LustreError: 307537:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9260.154238] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 9260.408231] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9264.190303] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing set_default_debug -1 all [ 9269.618436] Lustre: DEBUG MARKER: oleg217-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 9272.923778] Lustre: DEBUG MARKER: oleg217-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 9279.288195] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 9279.637574] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9284.511258] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing set_default_debug -1 all [ 9289.193723] Lustre: DEBUG MARKER: oleg217-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 9294.809754] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 9294.869819] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 9294.979831] Lustre: lustre-OST0001: new disk, initializing [ 9294.986957] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 9299.751378] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing set_default_debug -1 all [ 9301.595027] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 9301.599478] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 9301.654206] LustreError: 310487:0:(osp_precreate.c:969:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [ 9301.662220] LustreError: 310487:0:(osp_precreate.c:969:osp_precreate_cleanup_orphans()) Skipped 1 previous similar message [ 9302.692204] LustreError: 310491:0:(osp_precreate.c:969:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [ 9302.730148] LustreError: 310491:0:(osp_precreate.c:969:osp_precreate_cleanup_orphans()) Skipped 1 previous similar message [ 9303.714368] LustreError: 310487:0:(osp_precreate.c:969:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [ 9305.847844] LustreError: 310491:0:(osp_precreate.c:969:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [ 9305.858792] LustreError: 310491:0:(osp_precreate.c:969:osp_precreate_cleanup_orphans()) Skipped 4 previous similar messages [ 9308.372951] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 9308.597843] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 9309.990458] LustreError: 310491:0:(osp_precreate.c:969:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [ 9309.998910] LustreError: 310491:0:(osp_precreate.c:969:osp_precreate_cleanup_orphans()) Skipped 7 previous similar messages [ 9311.950833] Lustre: DEBUG MARKER: oleg217-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff9912d114b800.ost_server_uuid 50 [ 9313.342226] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff9912d114b800.ost_server_uuid in FULL state after 0 sec [ 9317.209249] Lustre: DEBUG MARKER: oleg217-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff9912d114b800.ost_server_uuid 50 [ 9318.193904] LustreError: 310487:0:(osp_precreate.c:969:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [ 9318.208283] LustreError: 310487:0:(osp_precreate.c:969:osp_precreate_cleanup_orphans()) Skipped 14 previous similar messages [ 9318.769119] Lustre: DEBUG MARKER: osc.lustre-OST0001-osc-ffff9912d114b800.ost_server_uuid in FULL state after 0 sec [ 9320.312822] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 9335.477602] Lustre: lustre-OST0001: Not available for connect from 192.168.202.17@tcp (stopping) [ 9335.785776] LustreError: lustre-OST0001-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 9335.789676] Lustre: lustre-OST0001-osc-MDT0000: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9335.835586] LustreError: Skipped 1 previous similar message [ 9339.662138] LustreError: 311348:0:(obd_class.h:479:obd_check_dev()) Device 25 not setup [ 9339.664755] LustreError: 311348:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 9339.888489] Lustre: server umount lustre-OST0001 complete [ 9340.545892] LustreError: 311242:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 192.168.202.17@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9342.950529] LustreError: 309137:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9342.972526] LustreError: 309137:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 9345.511701] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 9345.532332] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9345.553806] Lustre: Skipped 1 previous similar message [ 9345.561344] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9345.572140] Lustre: Skipped 4 previous similar messages [ 9348.072192] LustreError: 311242:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9348.078902] LustreError: 311242:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 9351.030751] Lustre: server umount lustre-OST0000 complete [ 9357.793360] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 9357.805858] LustreError: Skipped 1 previous similar message [ 9357.810208] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9357.824661] Lustre: Skipped 1 previous similar message [ 9360.532617] LustreError: 311752:0:(obd_class.h:479:obd_check_dev()) Device 28 not setup [ 9360.543397] LustreError: 311752:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 9360.643632] Lustre: server umount lustre-MDT0000 complete [ 9363.425377] LustreError: 307537:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9363.461096] LustreError: 307537:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 9364.568262] LustreError: 307517:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763343679 with bad export cookie 12128989625225176875 [ 9364.570250] LustreError: MGC192.168.202.117@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9364.592927] LustreError: 307517:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 9364.955083] Lustre: server umount lustre-MDT0001 complete [ 9373.556882] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing unload_modules_local [ 9375.756068] Key type lgssc unregistered [ 9376.020162] LNet: 312485:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 9376.030293] LNetError: 312485:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 9376.050171] LNet: Removed LNI 192.168.202.117@tcp [ 9376.886898] Key type .llcrypt unregistered [ 9376.888261] Key type ._llcrypt unregistered [ 9396.391672] Key type ._llcrypt registered [ 9396.392862] Key type .llcrypt registered [ 9396.489780] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing load_modules_local [ 9397.359291] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 9397.391160] alg: No test for adler32 (adler32-zlib) [ 9398.419532] Lustre: Lustre: Build Version: 2.16.61_54_g3c0e0e6 [ 9398.583889] LNet: Added LNI 192.168.202.117@tcp [8/256/0/180] [ 9400.224219] Key type lgssc registered [ 9401.122586] Lustre: Echo OBD driver; http://www.lustre.org/ [ 9407.189074] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 9407.200932] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9408.558668] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9411.835765] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing set_default_debug -1 all [ 9416.674037] Lustre: DEBUG MARKER: oleg217-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 9418.730408] LustreError: 313883:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9418.751919] LustreError: 313883:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 9422.894494] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 9423.182692] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9426.978419] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing set_default_debug -1 all [ 9435.596933] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 9435.949981] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9441.377260] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing set_default_debug -1 all [ 9446.901735] Lustre: DEBUG MARKER: oleg217-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 9448.240567] LustreError: 315421:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9453.542406] LustreError: 315420:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9454.376962] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 9454.572456] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:65) [ 9454.655497] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 9459.748240] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:33) [ 9461.394754] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing set_default_debug -1 all [ 9466.647936] Lustre: DEBUG MARKER: oleg217-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid [ 9472.190575] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing wait_import_state (FULL|IDLE) os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [ 9472.374389] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 9577.443294] LustreError: lustre-OST0001-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 9577.461472] Lustre: lustre-OST0001-osc-MDT0000: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9577.483687] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 9577.494174] Lustre: Skipped 1 previous similar message [ 9580.516709] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 9580.529653] Lustre: Skipped 1 previous similar message [ 9582.274121] LustreError: 317294:0:(obd_class.h:479:obd_check_dev()) Device 27 not setup [ 9582.399528] Lustre: server umount lustre-OST0001 complete [ 9585.635248] LustreError: 316561:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9585.655089] LustreError: 316561:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 9587.685482] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 9587.690064] LustreError: Skipped 1 previous similar message [ 9587.692281] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9587.699001] Lustre: Skipped 1 previous similar message [ 9587.704548] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9590.767394] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9590.770625] LustreError: 315420:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9590.776425] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9590.811141] LustreError: 315420:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 9593.104135] LustreError: 317497:0:(obd_class.h:479:obd_check_dev()) Device 23 not setup [ 9593.121035] LustreError: 317497:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 9593.266119] Lustre: server umount lustre-OST0000 complete [ 9597.409593] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 9597.413233] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9597.459969] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9597.462710] Lustre: Skipped 1 previous similar message [ 9603.227160] LustreError: 317699:0:(obd_class.h:479:obd_check_dev()) Device 10 not setup [ 9603.230116] LustreError: 317699:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 9603.303920] Lustre: server umount lustre-MDT0000 complete [ 9606.121401] LustreError: 313864:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9606.957911] LustreError: 315442:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763343921 with bad export cookie 12645577640118702297 [ 9606.963827] LustreError: MGC192.168.202.117@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9606.977750] LustreError: 315442:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 9607.139735] LustreError: 317900:0:(obd_class.h:479:obd_check_dev()) Device 18 not setup [ 9607.153280] LustreError: 317900:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 9607.352854] Lustre: server umount lustre-MDT0001 complete [ 9616.491406] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing unload_modules_local [ 9619.384973] Key type lgssc unregistered [ 9619.762403] LNet: 318432:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 9619.772890] LNetError: 318432:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 9619.798485] LNet: Removed LNI 192.168.202.117@tcp [ 9620.814783] Key type .llcrypt unregistered [ 9620.816278] Key type ._llcrypt unregistered [ 9664.337697] Key type ._llcrypt registered [ 9664.343733] Key type .llcrypt registered [ 9664.439496] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing load_modules_local [ 9665.445793] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 9665.503309] alg: No test for adler32 (adler32-zlib) [ 9666.593914] Lustre: Lustre: Build Version: 2.16.61_54_g3c0e0e6 [ 9666.889278] LNet: Added LNI 192.168.202.117@tcp [8/256/0/180] [ 9668.608206] Key type lgssc registered [ 9669.781663] Lustre: Echo OBD driver; http://www.lustre.org/ [ 9678.958608] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 9678.984243] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9680.454754] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9684.246885] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing set_default_debug -1 all [ 9690.593169] LustreError: 320704:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9690.600389] LustreError: 320704:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 9691.427344] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 9691.620721] Lustre: lustre-MDT0001: Not available for connect from 0@lo (not set up) [ 9691.800492] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9695.154427] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing set_default_debug -1 all [ 9697.779988] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 9703.772921] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 9704.075240] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9707.119155] LustreError: 322161:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9707.145172] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:97) [ 9709.223311] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing set_default_debug -1 all [ 9712.624943] LustreError: 322161:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9716.673661] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 9716.709872] LustreError: 322161:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9716.744255] LustreError: 322161:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 9716.948051] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 9721.344086] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing set_default_debug -1 all [ 9722.388706] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:65) [ 9729.221912] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 9736.456852] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 9753.058956] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9753.064960] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 9753.069162] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9753.084874] Lustre: Skipped 3 previous similar messages [ 9757.745129] LustreError: 324115:0:(obd_class.h:479:obd_check_dev()) Device 10 not setup [ 9757.824936] Lustre: server umount lustre-MDT0000 complete [ 9758.181735] LustreError: 322705:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9758.201374] LustreError: 322705:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 9761.071257] LustreError: 321808:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763344075 with bad export cookie 8154698228466316848 [ 9761.081694] LustreError: MGC192.168.202.117@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9761.087834] LustreError: 321808:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 9761.206231] LustreError: 324317:0:(obd_class.h:479:obd_check_dev()) Device 18 not setup [ 9761.213207] LustreError: 324317:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 9761.377335] Lustre: server umount lustre-MDT0001 complete [ 9775.120518] LustreError: 324518:0:(obd_class.h:479:obd_check_dev()) Device 23 not setup [ 9775.124414] LustreError: 324518:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 9775.226569] Lustre: server umount lustre-OST0000 complete [ 9788.751320] LustreError: 324720:0:(obd_class.h:479:obd_check_dev()) Device 2 not setup [ 9788.753757] LustreError: 324720:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 9788.899320] Lustre: server umount lustre-OST0001 complete [ 9803.053396] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing load_modules_local [ 9811.833207] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9812.175601] LustreError: 325898:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9812.246929] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9815.397392] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing set_default_debug -1 all [ 9823.398552] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 9824.012908] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9827.647629] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing set_default_debug -1 all [ 9830.450204] Lustre: 327006:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 9836.319926] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 9836.518759] LustreError: 327360:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9836.542511] LustreError: 327360:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 9838.608290] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:129) [ 9841.566385] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing set_default_debug -1 all [ 9849.141820] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 9849.381694] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 9849.384325] Lustre: Skipped 1 previous similar message [ 9850.886874] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:97) [ 9854.978898] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing set_default_debug -1 all [ 9861.510923] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 9864.680116] Lustre: 328851:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 9874.916380] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9874.926190] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9874.934126] Lustre: Skipped 3 previous similar messages [ 9874.937797] Lustre: Skipped 5 previous similar messages [ 9878.833615] LustreError: 328956:0:(obd_class.h:479:obd_check_dev()) Device 10 not setup [ 9878.843913] LustreError: 328956:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 9878.943531] Lustre: server umount lustre-MDT0000 complete [ 9880.054383] LustreError: 325897:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9880.064162] LustreError: 325897:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 6 previous similar messages [ 9882.719137] LustreError: 325878:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763344197 with bad export cookie 8154698228466318549 [ 9882.723053] LustreError: MGC192.168.202.117@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9882.727725] LustreError: 325878:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 9883.152688] Lustre: server umount lustre-MDT0001 complete [ 9897.100865] LustreError: 329358:0:(obd_class.h:479:obd_check_dev()) Device 23 not setup [ 9897.103556] LustreError: 329358:0:(obd_class.h:479:obd_check_dev()) Skipped 15 previous similar messages [ 9897.227656] Lustre: server umount lustre-OST0000 complete [ 9901.092029] Lustre: 319966:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1763344199/real 1763344199] req@ffff89e8f73bdc00 x1849000178329728/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1763344215 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 9901.127757] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9901.448224] Lustre: server umount lustre-OST0001 complete [ 9905.069827] Lustre: DEBUG MARKER: SKIP: conf-sanity test_114 skipping SLOW test 114 [ 9906.625375] Lustre: DEBUG MARKER: SKIP: conf-sanity test_115 skipping excluded test 115 [ 9908.349160] Lustre: DEBUG MARKER: == conf-sanity test 116: big size MDT support ============ 20:50:21 (1763344221) [ 9928.650683] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing load_modules_local [ 9937.387760] loop0: detected capacity change from 0 to 1099511627776 [ 9937.820810] SGI XFS with ACLs, security attributes, quota, no debug enabled [ 9937.827100] *** VALIDATE xfs *** [ 9937.848668] XFS (loop0): Mounting V5 Filesystem [ 9937.876082] XFS (loop0): Ending clean mount [ 9938.886982] loop1: detected capacity change from 0 to 18691697672192 [10004.200912] LDISKFS-fs (loop1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [10020.219233] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing set_hostid [10028.207262] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing load_modules_local [10036.412726] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [10041.597100] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [10046.989031] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [10052.913664] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [10059.168593] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [10059.271860] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10059.586876] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [10059.617924] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [10059.686924] Lustre: lustre-MDT0000: new disk, initializing [10059.813193] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10059.849914] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [10063.563081] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing set_default_debug -1 all [10071.153971] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [10071.224338] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [10071.293065] Lustre: 333944:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [10071.348783] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [10071.351351] Lustre: Skipped 1 previous similar message [10071.422611] Lustre: lustre-MDT0001: new disk, initializing [10071.559420] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [10071.571587] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [10074.267612] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing set_default_debug -1 all [10079.988470] Lustre: DEBUG MARKER: oleg217-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [10082.808996] Lustre: DEBUG MARKER: oleg217-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [10088.491691] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [10088.572980] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [10088.881347] Lustre: lustre-OST0000: new disk, initializing [10088.884243] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [10089.922177] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [10089.927881] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [10090.035229] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [10093.819607] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing set_default_debug -1 all [10099.786244] Lustre: DEBUG MARKER: oleg217-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [10103.435347] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [10103.619502] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [10107.044295] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [10107.261371] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [10110.436992] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [10110.449214] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10110.460018] LustreError: Skipped 1 previous similar message [10110.464742] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10110.464750] Lustre: Skipped 1 previous similar message [10110.489347] Lustre: Skipped 1 previous similar message [10115.569815] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10115.571995] Lustre: Skipped 1 previous similar message [10123.232211] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [10123.268891] LustreError: 336881:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [10123.271237] LustreError: 336881:0:(obd_class.h:479:obd_check_dev()) Skipped 9 previous similar messages [10123.412858] Lustre: server umount lustre-OST0000 complete [10127.841340] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [10127.843617] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10127.854413] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10127.856884] Lustre: Skipped 3 previous similar messages [10132.744072] Lustre: server umount lustre-MDT0000 complete [10136.061456] LustreError: 333956:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10136.079310] LustreError: 333956:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [10136.099846] LustreError: 334724:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763344450 with bad export cookie 8154698228466320103 [10136.104892] LustreError: MGC192.168.202.117@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10136.117605] LustreError: 334724:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [10136.691201] Lustre: server umount lustre-MDT0001 complete [10143.728337] XFS (loop0): Unmounting Filesystem [10147.827177] Lustre: DEBUG MARKER: == conf-sanity test 117: lctl get_param return errors properly ========================================================== 20:54:21 (1763344461) [10153.937849] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10154.307316] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10154.310270] Lustre: Skipped 2 previous similar messages [10157.383530] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing set_default_debug -1 all [10163.897162] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [10167.118906] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing set_default_debug -1 all [10171.583592] Lustre: DEBUG MARKER: oleg217-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [10174.891431] Lustre: DEBUG MARKER: oleg217-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [10182.697278] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [10189.165967] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing set_default_debug -1 all [10194.596387] Lustre: DEBUG MARKER: oleg217-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [10209.250644] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [10209.255307] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10209.265984] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10209.270017] Lustre: Skipped 2 previous similar messages [10209.274797] Lustre: Skipped 3 previous similar messages [10212.032547] LustreError: 340447:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [10212.039112] LustreError: 340447:0:(obd_class.h:479:obd_check_dev()) Skipped 17 previous similar messages [10212.144718] Lustre: server umount lustre-OST0000 complete [10215.393453] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [10220.826389] Lustre: server umount lustre-MDT0000 complete [10224.270660] LustreError: 338086:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763344538 with bad export cookie 8154698228466320985 [10224.273916] LustreError: MGC192.168.202.117@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10224.311097] LustreError: 338086:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 1 previous similar message [10224.750953] Lustre: server umount lustre-MDT0001 complete [10232.273831] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing unload_modules_local [10234.489740] Key type lgssc unregistered [10234.810717] LNet: 341383:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [10234.820141] LNetError: 341383:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [10234.833644] LNet: Removed LNI 192.168.202.117@tcp [10235.517162] Key type .llcrypt unregistered [10235.519032] Key type ._llcrypt unregistered [10258.063431] Key type ._llcrypt registered [10258.065042] Key type .llcrypt registered [10258.133583] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing load_modules_local [10258.800050] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [10258.808901] alg: No test for adler32 (adler32-zlib) [10259.844605] Lustre: Lustre: Build Version: 2.16.61_54_g3c0e0e6 [10260.091431] LNet: Added LNI 192.168.202.117@tcp [8/256/0/180] [10261.848220] Key type lgssc registered [10262.804803] Lustre: Echo OBD driver; http://www.lustre.org/ [10269.482492] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [10269.506609] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10270.907921] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10273.934932] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing set_default_debug -1 all [10280.424959] LustreError: 342909:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10280.445457] LustreError: 342909:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [10281.726701] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [10282.008597] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10285.546675] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing set_default_debug -1 all [10290.078534] Lustre: DEBUG MARKER: oleg217-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [10292.957994] Lustre: DEBUG MARKER: oleg217-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [10299.090906] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [10299.353315] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10304.269661] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing set_default_debug -1 all [10309.142417] Lustre: DEBUG MARKER: oleg217-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [10325.129977] Lustre: lustre-MDT0001: Not available for connect from 192.168.202.17@tcp (stopping) [10325.988137] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10326.003769] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [10326.008502] Lustre: Skipped 1 previous similar message [10330.233713] Lustre: lustre-MDT0001: Not available for connect from 192.168.202.17@tcp (stopping) [10330.259529] LustreError: 345225:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [10330.318730] Lustre: server umount lustre-MDT0001 complete [10331.107341] LustreError: 342910:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10331.114317] LustreError: 342910:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [10331.909587] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [10335.378449] LustreError: 344863:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.202.17@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [10337.146549] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [10337.220416] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [10337.226839] Lustre: Found index 1 for lustre-MDT0001, updating log [10337.252511] Lustre: 342898:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [10337.500638] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10340.125334] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing set_default_debug -1 all [10342.893392] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [10347.515114] LustreError: 346263:0:(obd_mount.c:188:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [10347.519940] LustreError: 346263:0:(tgt_mount.c:669:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [10347.537543] LustreError: 342901:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [10347.589731] LustreError: 346268:0:(obd_config.c:1997:class_config_llog_handler()) MGC192.168.202.117@tcp: cfg command failed: rc = -17 [10347.598885] Lustre: cmd=cf001 0:lustre-MDT0001-osp-MDT0000 1:osp 2:lustre-MDT0000-mdtlov_UUID [10347.598885] [10647.702757] Lustre: lustre-MDT0001: Not available for connect from 192.168.202.17@tcp (stopping) [10650.088296] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10650.124752] Lustre: Skipped 1 previous similar message [10650.127824] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [10651.600603] LustreError: 354457:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [10651.604138] LustreError: 354457:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [10651.858206] Lustre: server umount lustre-MDT0001 complete [10652.804792] LustreError: 347869:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.202.17@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [10652.838589] LustreError: 347869:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [10655.223508] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [10657.926666] LustreError: 342906:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.202.17@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [10657.952177] LustreError: 342906:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [10662.351267] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [10662.411468] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [10662.414529] Lustre: Found index 1 for lustre-MDT0001, updating log [10662.459298] Lustre: 342898:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [10662.705778] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10665.697976] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing set_default_debug -1 all [10668.019199] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [10670.085626] LustreError: 355436:0:(obd_mount.c:188:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [10670.103989] LustreError: 355436:0:(tgt_mount.c:669:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [10670.111155] LustreError: 342901:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [10670.114088] LustreError: 342901:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [10670.200826] LustreError: 355440:0:(obd_config.c:689:class_setup()) Device 8 already setup (type osp) [10670.203546] LustreError: 355440:0:(obd_config.c:1997:class_config_llog_handler()) MGC192.168.202.117@tcp: cfg command failed: rc = -17 [10670.216934] Lustre: cmd=cf003 0:lustre-MDT0001-osp-MDT0000 1:lustre-MDT0001_UUID 2:192.168.202.117@tcp [10670.216934] [10973.155400] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10973.173713] Lustre: Skipped 1 previous similar message [10973.183953] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [10973.186444] Lustre: Skipped 2 previous similar messages [10976.958600] LustreError: 363775:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [10976.974744] LustreError: 363775:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [10977.151240] Lustre: server umount lustre-MDT0001 complete [10978.279326] LustreError: 347869:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10978.304632] LustreError: 347869:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [10979.487322] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [10986.106691] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [10986.268059] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [10986.271907] Lustre: Found index 1 for lustre-MDT0001, updating log [10986.327953] Lustre: 342898:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [10986.670652] Lustre: lustre-MDT0001: Imperative Recovery enabled, recovery window shrunk from 60-180 down to 60-180 [10989.958885] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing set_default_debug -1 all [10992.111717] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [10994.168054] LustreError: 364755:0:(obd_mount.c:188:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [10994.175846] LustreError: 364755:0:(tgt_mount.c:669:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [10994.190468] LustreError: 342901:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [10994.193647] LustreError: 342901:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [10994.305279] LustreError: 364758:0:(lod_lov.c:207:lod_add_device()) lustre-MDT0001-osp-MDT0000: cannot connect to next dev lustre-MDT0001_UUID (-114) [10994.314822] LustreError: 364758:0:(obd_config.c:1997:class_config_llog_handler()) MGC192.168.202.117@tcp: cfg command failed: rc = -114 [10994.318782] Lustre: cmd=cf014 0:lustre-MDT0000-mdtlov 1:lustre-MDT0001_UUID 2:1 3:1 [10994.318782] [11309.539744] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11309.555514] Lustre: Skipped 1 previous similar message [11309.561394] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11309.565300] Lustre: Skipped 2 previous similar messages [11314.145314] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [11314.153279] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11314.159895] Lustre: Skipped 1 previous similar message [11315.501978] LustreError: 372910:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup [11315.506332] LustreError: 372910:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [11315.567344] Lustre: server umount lustre-MDT0000 complete [11318.787760] LustreError: 342890:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763345633 with bad export cookie 5501190865812141334 [11318.788898] LustreError: MGC192.168.202.117@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11318.792209] LustreError: 342890:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [11319.780603] LustreError: 344863:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11319.808734] LustreError: 344863:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 4 previous similar messages [11328.937359] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [11328.939610] Lustre: Skipped 5 previous similar messages [11333.088706] Lustre: lustre-MDT0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [11333.144212] LustreError: 373111:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [11333.147529] LustreError: 373111:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [11333.269830] Lustre: server umount lustre-MDT0001 complete [11336.861402] Lustre: server umount lustre-OST0000 complete [11345.195658] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing set_hostid [11353.496879] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing load_modules_local [11363.212120] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [11368.945521] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [11374.965728] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [11380.646942] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [11386.793784] Lustre: DEBUG MARKER: == conf-sanity test 120: cross-target rename should not create bad symlinks ========================================================== 21:15:00 (1763345700) [11392.785198] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [11392.834812] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11393.063446] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [11393.096634] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [11393.175895] Lustre: lustre-MDT0000: new disk, initializing [11393.242792] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11393.272409] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [11396.603681] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing set_default_debug -1 all [11405.519568] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [11405.600117] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [11405.678855] Lustre: 375854:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [11405.732969] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [11405.735719] Lustre: Skipped 1 previous similar message [11405.802213] Lustre: lustre-MDT0001: new disk, initializing [11405.907628] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11405.958951] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [11405.972787] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [11409.863537] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing set_default_debug -1 all [11416.739599] Lustre: DEBUG MARKER: oleg217-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [11419.827556] Lustre: DEBUG MARKER: oleg217-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [11425.435753] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [11425.511361] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [11425.821144] Lustre: lustre-OST0000: new disk, initializing [11425.832079] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [11425.940397] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [11431.627224] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing set_default_debug -1 all [11432.505736] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [11432.513196] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [11432.690818] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [11439.322385] Lustre: DEBUG MARKER: oleg217-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [11446.755762] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [11446.764729] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11446.779382] Lustre: Skipped 1 previous similar message [11446.787289] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11451.462699] LustreError: 378294:0:(obd_class.h:479:obd_check_dev()) Device 22 not setup [11451.476652] LustreError: 378294:0:(obd_class.h:479:obd_check_dev()) Skipped 11 previous similar messages [11451.625158] Lustre: server umount lustre-MDT0000 complete [11453.411658] LustreError: 375865:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11453.433790] LustreError: 375865:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 4 previous similar messages [11454.823548] LustreError: 376643:0:(osp_object.c:617:osp_attr_get()) lustre-MDT0000-osp-MDT0001: osp_attr_get update error [0x200000401:0x2:0x0]: rc = -5 [11454.829846] LustreError: 375845:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763345769 with bad export cookie 5501190865812143301 [11454.833244] LustreError: MGC192.168.202.117@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11454.849920] LustreError: 375845:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [11455.120905] Lustre: server umount lustre-MDT0001 complete [11468.670220] LustreError: 378696:0:(obd_class.h:479:obd_check_dev()) Device 2 not setup [11468.672534] LustreError: 378696:0:(obd_class.h:479:obd_check_dev()) Skipped 11 previous similar messages [11468.875590] Lustre: server umount lustre-OST0000 complete [11497.682857] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing set_hostid [11507.813423] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing load_modules_local [11517.051835] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [11523.281468] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [11528.791304] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [11535.455781] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [11537.305166] Lustre: DEBUG MARKER: == conf-sanity test complete, duration 11289 sec ========= 21:17:30 (1763345850) [11539.075241] Lustre: DEBUG MARKER: === conf-sanity: start cleanup 21:17:32 (1763345852) === [11542.370872] Lustre: DEBUG MARKER: === conf-sanity: finish cleanup 21:17:36 (1763345856) === [11579.489706] Lustre: DEBUG MARKER: oleg217-server.virtnet: executing unload_modules_local [11582.193282] Key type lgssc unregistered [11582.477840] LNet: 383809:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [11582.490308] LNetError: 383809:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [11582.510655] LNet: Removed LNI 192.168.202.117@tcp [11583.390182] Key type .llcrypt unregistered [11583.391843] Key type ._llcrypt unregistered