[ 0.000000] Linux version 4.18.0rh8.10-debug (green@maintenance) (gcc version 8.5.0 20210514 (Red Hat 8.5.0-26) (GCC)) #2 SMP Mon Jul 14 01:24:22 EDT 2025 [ 0.000000] Command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers' [ 0.000000] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256 [ 0.000000] x86/fpu: Enabled xstate features 0x7, context size is 832 bytes, using 'standard' format. [ 0.000000] signal: max sigframe size: 1776 [ 0.000000] BIOS-provided physical RAM map: [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x00000000bffcdfff] usable [ 0.000000] BIOS-e820: [mem 0x00000000bffce000-0x00000000bfffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x0000000146dfffff] usable [ 0.000000] NX (Execute Disable) protection: active [ 0.000000] SMBIOS 2.8 present. [ 0.000000] DMI: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.16.2-1.fc38 04/01/2014 [ 0.000000] Hypervisor detected: KVM [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00 [ 0.000000] kvm-clock: using sched offset of 547231835 cycles [ 0.000000] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns [ 0.000000] tsc: Detected 2399.998 MHz processor [ 0.000000] last_pfn = 0x146e00 max_arch_pfn = 0x400000000 [ 0.000000] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT [ 0.000000] last_pfn = 0xbffce max_arch_pfn = 0x400000000 [ 0.000000] found SMP MP-table at [mem 0x000f5b30-0x000f5b3f] [ 0.000000] RAMDISK: [mem 0xbcc54000-0xbffbffff] [ 0.000000] ACPI: Early table checksum verification disabled [ 0.000000] ACPI: RSDP 0x00000000000F5950 000014 (v00 BOCHS ) [ 0.000000] ACPI: RSDT 0x00000000BFFE1BB7 000034 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACP 0x00000000BFFE1A53 000074 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: DSDT 0x00000000BFFE0040 001A13 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACS 0x00000000BFFE0000 000040 [ 0.000000] ACPI: APIC 0x00000000BFFE1AC7 000090 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: HPET 0x00000000BFFE1B57 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: WAET 0x00000000BFFE1B8F 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: Reserving FACP table memory at [mem 0xbffe1a53-0xbffe1ac6] [ 0.000000] ACPI: Reserving DSDT table memory at [mem 0xbffe0040-0xbffe1a52] [ 0.000000] ACPI: Reserving FACS table memory at [mem 0xbffe0000-0xbffe003f] [ 0.000000] ACPI: Reserving APIC table memory at [mem 0xbffe1ac7-0xbffe1b56] [ 0.000000] ACPI: Reserving HPET table memory at [mem 0xbffe1b57-0xbffe1b8e] [ 0.000000] ACPI: Reserving WAET table memory at [mem 0xbffe1b8f-0xbffe1bb6] [ 0.000000] No NUMA configuration found [ 0.000000] Faking a node at [mem 0x0000000000000000-0x0000000146dfffff] [ 0.000000] NODE_DATA(0) allocated [mem 0x1465a3000-0x1465cdfff] [ 0.000000] Reserving 256MB of memory at 2752MB for crashkernel (System RAM: 4205MB) [ 0.000000] Zone ranges: [ 0.000000] DMA [mem 0x0000000000001000-0x0000000000ffffff] [ 0.000000] DMA32 [mem 0x0000000001000000-0x00000000ffffffff] [ 0.000000] Normal [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Device empty [ 0.000000] Movable zone start for each node [ 0.000000] Early memory node ranges [ 0.000000] node 0: [mem 0x0000000000001000-0x000000000009efff] [ 0.000000] node 0: [mem 0x0000000000100000-0x00000000bffcdfff] [ 0.000000] node 0: [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Zeroed struct page in unavailable ranges: 4756 pages [ 0.000000] Initmem setup node 0 [mem 0x0000000000001000-0x0000000146dfffff] [ 0.000000] ACPI: PM-Timer IO Port: 0x608 [ 0.000000] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1]) [ 0.000000] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23 [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level) [ 0.000000] Using ACPI (MADT) for SMP configuration information [ 0.000000] ACPI: HPET id: 0x8086a201 base: 0xfed00000 [ 0.000000] TSC deadline timer available [ 0.000000] smpboot: Allowing 4 CPUs, 0 hotplug CPUs [ 0.000000] kvm-guest: KVM setup pv remote TLB flush [ 0.000000] kvm-guest: setup PV sched yield [ 0.000000] PM: Registered nosave memory: [mem 0x00000000-0x00000fff] [ 0.000000] PM: Registered nosave memory: [mem 0x0009f000-0x0009ffff] [ 0.000000] PM: Registered nosave memory: [mem 0x000a0000-0x000effff] [ 0.000000] PM: Registered nosave memory: [mem 0x000f0000-0x000fffff] [ 0.000000] PM: Registered nosave memory: [mem 0xbffce000-0xbfffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xc0000000-0xfeffbfff] [ 0.000000] PM: Registered nosave memory: [mem 0xfeffc000-0xfeffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xff000000-0xfffbffff] [ 0.000000] PM: Registered nosave memory: [mem 0xfffc0000-0xffffffff] [ 0.000000] [mem 0xc0000000-0xfeffbfff] available for PCI devices [ 0.000000] Booting paravirtualized kernel on KVM [ 0.000000] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns [ 0.000000] setup_percpu: NR_CPUS:8192 nr_cpumask_bits:4 nr_cpu_ids:4 nr_node_ids:1 [ 0.000000] percpu: Embedded 63 pages/cpu s221184 r8192 d28672 u524288 [ 0.000000] kvm-guest: PV spinlocks enabled [ 0.000000] PV qspinlock hash table entries: 256 (order: 0, 4096 bytes, linear) [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 1059606 [ 0.000000] Policy zone: Normal [ 0.000000] Kernel command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] Specific versions of hardware are certified with Red Hat Enterprise Linux 8. Please see the list of hardware certified with Red Hat Enterprise Linux 8 at https://catalog.redhat.com. [ 0.000000] audit: disabled (until reboot) [ 0.000000] software IO TLB: area num 4. [ 0.000000] Memory: 2864520K/4306352K available (18435K kernel code, 11221K rwdata, 7248K rodata, 2908K init, 18040K bss, 524584K reserved, 0K cma-reserved) [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1 [ 0.000000] kmemleak: Kernel memory leak detector disabled [ 0.000000] ftrace: allocating 41240 entries in 162 pages [ 0.000000] ftrace: allocated 162 pages with 3 groups [ 0.000000] rcu: Hierarchical RCU implementation. [ 0.000000] rcu: RCU event tracing is enabled. [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=8192 to nr_cpu_ids=4. [ 0.000000] rcu: RCU callback double-/use-after-free debug enabled. [ 0.000000] Rude variant of Tasks RCU enabled. [ 0.000000] Tracing variant of Tasks RCU enabled. [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies. [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4 [ 0.000000] NR_IRQS: 524544, nr_irqs: 456, preallocated irqs: 16 [ 0.000000] random: get_random_bytes called from start_kernel+0x622/0x9a8 with crng_init=0 [ 0.001000] Console: colour *CGA 80x25 [ 0.001000] printk: console [ttyS1] enabled [ 0.001000] ACPI: Core revision 20220331 [ 0.001000] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns [ 0.001010] APIC: Switch to symmetric I/O mode setup [ 0.002000] x2apic enabled [ 0.002000] Switched APIC routing to physical x2apic. [ 0.002012] kvm-guest: setup PV IPIs [ 0.005553] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1 [ 0.006000] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 0.006021] Calibrating delay loop (skipped) preset value.. 4799.99 BogoMIPS (lpj=2399998) [ 0.007011] pid_max: default: 32768 minimum: 301 [ 0.008137] LSM: Security Framework initializing [ 0.009043] Yama: becoming mindful. [ 0.010025] SELinux: Initializing. [ 0.011056] *** VALIDATE selinux *** [ 0.019815] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, vmalloc) [ 0.024145] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, vmalloc) [ 0.026041] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.027094] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.028119] *** VALIDATE tmpfs *** [ 0.030027] *** VALIDATE proc *** [ 0.031238] *** VALIDATE cgroup *** [ 0.032006] *** VALIDATE cgroup2 *** [ 0.034239] x86/cpu: User Mode Instruction Prevention (UMIP) activated [ 0.035149] Last level iTLB entries: 4KB 0, 2MB 0, 4MB 0 [ 0.036008] Last level dTLB entries: 4KB 0, 2MB 0, 4MB 0, 1GB 0 [ 0.037028] Spectre V2 : User space: Vulnerable [ 0.038000] Speculative Store Bypass: Vulnerable [ 0.040415] debug: unmapping init [mem 0xffffffff9bc59000-0xffffffff9bc60fff] [ 0.043075] smpboot: CPU0: Intel(R) Xeon(R) CPU E5-2695 v2 @ 2.40GHz (family: 0x6, model: 0x3e, stepping: 0x4) [ 0.044561] Performance Events: IvyBridge events, full-width counters, Intel PMU driver. [ 0.045015] ... version: 2 [ 0.046007] ... bit width: 48 [ 0.047007] ... generic registers: 4 [ 0.048008] ... value mask: 0000ffffffffffff [ 0.049007] ... max period: 00007fffffffffff [ 0.050007] ... fixed-purpose events: 3 [ 0.050982] ... event mask: 000000070000000f [ 0.051264] rcu: Hierarchical SRCU implementation. [ 0.053583] smp: Bringing up secondary CPUs ... [ 0.054622] x86: Booting SMP configuration: [ 0.055014] .... node #0, CPUs: #1 #2 #3 [ 0.073561] smp: Brought up 1 node, 4 CPUs [ 0.075009] smpboot: Max logical packages: 1 [ 0.076012] smpboot: Total of 4 processors activated (19199.98 BogoMIPS) [ 0.109020] node 0 deferred pages initialised in 30ms [ 0.112007] devtmpfs: initialized [ 0.113726] x86/mm: Memory block size: 128MB [ 0.115827] gcov: version magic: 0x41383552 [ 0.117066] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns [ 0.118057] futex hash table entries: 1024 (order: 4, 65536 bytes, vmalloc) [ 0.119211] pinctrl core: initialized pinctrl subsystem [ 0.120163] [ 0.120730] ************************************************************* [ 0.121009] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.122009] ** ** [ 0.123009] ** IOMMU DebugFS SUPPORT HAS BEEN ENABLED IN THIS KERNEL ** [ 0.124010] ** ** [ 0.125009] ** This means that this kernel is built to expose internal ** [ 0.126010] ** IOMMU data structures, which may compromise security on ** [ 0.127008] ** your system. ** [ 0.128010] ** ** [ 0.129011] ** If you see this message and you are not debugging the ** [ 0.130009] ** kernel, report this immediately to your vendor! ** [ 0.131010] ** ** [ 0.132010] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.133049] ************************************************************* [ 0.136326] NET: Registered protocol family 16 [ 0.138510] DMA: preallocated 512 KiB GFP_KERNEL pool for atomic allocations [ 0.141049] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations [ 0.143057] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations [ 0.147332] cpuidle: using governor menu [ 0.149863] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5 [ 0.152532] PCI: Using configuration type 1 for base access [ 0.155121] core: PMU erratum BJ122, BV98, HSD29 worked around, HT is on [ 0.158066] HugeTLB registered 1.00 GiB page size, pre-allocated 0 pages [ 0.159015] HugeTLB registered 2.00 MiB page size, pre-allocated 0 pages [ 0.172571] cryptd: max_cpu_qlen set to 1000 [ 0.174307] ACPI: Added _OSI(Module Device) [ 0.176010] ACPI: Added _OSI(Processor Device) [ 0.177013] ACPI: Added _OSI(3.0 _SCP Extensions) [ 0.179010] ACPI: Added _OSI(Processor Aggregator Device) [ 0.184346] ACPI: 1 ACPI AML tables successfully acquired and loaded [ 0.190638] ACPI: Interpreter enabled [ 0.191000] ACPI: PM: (supports S0 S3 S4 S5) [ 0.193007] ACPI: Using IOAPIC for interrupt routing [ 0.194202] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug [ 0.197386] ACPI: Enabled 2 GPEs in block 00 to 0F [ 0.209346] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff]) [ 0.212036] acpi PNP0A03:00: _OSC: OS supports [ASPM ClockPM Segments MSI HPX-Type3] [ 0.217019] acpi PNP0A03:00: _OSC: not requesting OS control; OS requires [ExtendedConfig ASPM ClockPM MSI] [ 0.222074] acpi PNP0A03:00: fail to add MMCONFIG information, can't access extended PCI configuration space under this bridge. [ 0.231355] acpiphp: Slot [2] registered [ 0.233177] acpiphp: Slot [3] registered [ 0.236251] acpiphp: Slot [4] registered [ 0.239593] acpiphp: Slot [5] registered [ 0.241140] acpiphp: Slot [6] registered [ 0.242124] acpiphp: Slot [7] registered [ 0.245333] acpiphp: Slot [8] registered [ 0.247093] acpiphp: Slot [9] registered [ 0.248080] acpiphp: Slot [10] registered [ 0.250300] acpiphp: Slot [11] registered [ 0.251078] acpiphp: Slot [12] registered [ 0.253074] acpiphp: Slot [13] registered [ 0.254078] acpiphp: Slot [14] registered [ 0.255084] acpiphp: Slot [15] registered [ 0.257081] acpiphp: Slot [16] registered [ 0.259122] acpiphp: Slot [17] registered [ 0.262101] acpiphp: Slot [18] registered [ 0.264082] acpiphp: Slot [19] registered [ 0.267173] acpiphp: Slot [20] registered [ 0.268194] acpiphp: Slot [21] registered [ 0.270090] acpiphp: Slot [22] registered [ 0.271117] acpiphp: Slot [23] registered [ 0.272105] acpiphp: Slot [24] registered [ 0.272932] acpiphp: Slot [25] registered [ 0.274103] acpiphp: Slot [26] registered [ 0.275094] acpiphp: Slot [27] registered [ 0.276087] acpiphp: Slot [28] registered [ 0.277077] acpiphp: Slot [29] registered [ 0.278114] acpiphp: Slot [30] registered [ 0.279093] acpiphp: Slot [31] registered [ 0.280059] PCI host bridge to bus 0000:00 [ 0.282016] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window] [ 0.284025] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window] [ 0.286046] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window] [ 0.288028] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window] [ 0.294026] pci_bus 0000:00: root bus resource [mem 0x180000000-0x1ffffffff window] [ 0.296023] pci_bus 0000:00: root bus resource [bus 00-ff] [ 0.298551] pci 0000:00:00.0: [8086:1237] type 00 class 0x060000 [ 0.304382] pci 0000:00:01.0: [8086:7000] type 00 class 0x060100 [ 0.308333] pci 0000:00:01.1: [8086:7010] type 00 class 0x010180 [ 0.324066] pci 0000:00:01.1: reg 0x20: [io 0xc320-0xc32f] [ 0.335203] pci 0000:00:01.1: legacy IDE quirk: reg 0x10: [io 0x01f0-0x01f7] [ 0.340021] pci 0000:00:01.1: legacy IDE quirk: reg 0x14: [io 0x03f6] [ 0.343028] pci 0000:00:01.1: legacy IDE quirk: reg 0x18: [io 0x0170-0x0177] [ 0.347031] pci 0000:00:01.1: legacy IDE quirk: reg 0x1c: [io 0x0376] [ 0.352634] pci 0000:00:01.3: [8086:7113] type 00 class 0x068000 [ 0.354704] pci 0000:00:01.3: quirk: [io 0x0600-0x063f] claimed by PIIX4 ACPI [ 0.357029] pci 0000:00:01.3: quirk: [io 0x0700-0x070f] claimed by PIIX4 SMB [ 0.359611] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 [ 0.364012] pci 0000:00:02.0: reg 0x10: [io 0xc300-0xc31f] [ 0.374014] pci 0000:00:02.0: reg 0x20: [mem 0xfebe4000-0xfebe7fff 64bit pref] [ 0.376015] pci 0000:00:02.0: reg 0x30: [mem 0xfeb80000-0xfebbffff pref] [ 0.381822] pci 0000:00:05.0: [1af4:1001] type 00 class 0x010000 [ 0.387015] pci 0000:00:05.0: reg 0x10: [io 0xc000-0xc07f] [ 0.391016] pci 0000:00:05.0: reg 0x14: [mem 0xfebc0000-0xfebc0fff] [ 0.400022] pci 0000:00:05.0: reg 0x20: [mem 0xfebe8000-0xfebebfff 64bit pref] [ 0.409542] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 [ 0.414015] pci 0000:00:06.0: reg 0x10: [io 0xc080-0xc0ff] [ 0.418014] pci 0000:00:06.0: reg 0x14: [mem 0xfebc1000-0xfebc1fff] [ 0.433014] pci 0000:00:06.0: reg 0x20: [mem 0xfebec000-0xfebeffff 64bit pref] [ 0.441450] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 [ 0.450029] pci 0000:00:07.0: reg 0x10: [io 0xc100-0xc17f] [ 0.457015] pci 0000:00:07.0: reg 0x14: [mem 0xfebc2000-0xfebc2fff] [ 0.480028] pci 0000:00:07.0: reg 0x20: [mem 0xfebf0000-0xfebf3fff 64bit pref] [ 0.500224] pci 0000:00:08.0: [1af4:1001] type 00 class 0x010000 [ 0.511020] pci 0000:00:08.0: reg 0x10: [io 0xc180-0xc1ff] [ 0.519018] pci 0000:00:08.0: reg 0x14: [mem 0xfebc3000-0xfebc3fff] [ 0.532017] pci 0000:00:08.0: reg 0x20: [mem 0xfebf4000-0xfebf7fff 64bit pref] [ 0.543320] pci 0000:00:09.0: [1af4:1001] type 00 class 0x010000 [ 0.548015] pci 0000:00:09.0: reg 0x10: [io 0xc200-0xc27f] [ 0.556014] pci 0000:00:09.0: reg 0x14: [mem 0xfebc4000-0xfebc4fff] [ 0.569015] pci 0000:00:09.0: reg 0x20: [mem 0xfebf8000-0xfebfbfff 64bit pref] [ 0.580436] pci 0000:00:0a.0: [1af4:1001] type 00 class 0x010000 [ 0.589013] pci 0000:00:0a.0: reg 0x10: [io 0xc280-0xc2ff] [ 0.596013] pci 0000:00:0a.0: reg 0x14: [mem 0xfebc5000-0xfebc5fff] [ 0.611014] pci 0000:00:0a.0: reg 0x20: [mem 0xfebfc000-0xfebfffff 64bit pref] [ 0.623609] ACPI: PCI: Interrupt link LNKA configured for IRQ 10 [ 0.625383] ACPI: PCI: Interrupt link LNKB configured for IRQ 10 [ 0.628321] ACPI: PCI: Interrupt link LNKC configured for IRQ 11 [ 0.630325] ACPI: PCI: Interrupt link LNKD configured for IRQ 11 [ 0.632204] ACPI: PCI: Interrupt link LNKS configured for IRQ 9 [ 0.639038] iommu: Default domain type: Passthrough [ 0.640000] SCSI subsystem initialized [ 0.640000] ACPI: bus type USB registered [ 0.640144] usbcore: registered new interface driver usbfs [ 0.642168] usbcore: registered new interface driver hub [ 0.644078] usbcore: registered new device driver usb [ 0.646141] pps_core: LinuxPPS API ver. 1 registered [ 0.648009] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti [ 0.651065] PTP clock support registered [ 0.653304] EDAC MC: Ver: 3.0.0 [ 0.655121] PCI: Using ACPI for IRQ routing [ 0.667849] NetLabel: Initializing [ 0.669007] NetLabel: domain hash size = 128 [ 0.670006] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO [ 0.673135] NetLabel: unlabeled traffic allowed by default [ 0.684120] vgaarb: loaded [ 0.686322] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0 [ 0.687010] hpet0: 3 comparators, 64-bit 100.000000 MHz counter [ 0.693141] clocksource: Switched to clocksource kvm-clock [ 0.834203] VFS: Disk quotas dquot_6.6.0 [ 0.835475] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) [ 0.837963] *** VALIDATE ramfs *** [ 0.839199] *** VALIDATE hugetlbfs *** [ 0.840763] pnp: PnP ACPI init [ 0.843106] pnp: PnP ACPI: found 6 devices [ 0.859678] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns [ 0.862865] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window] [ 0.864952] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window] [ 0.867026] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window] [ 0.869520] pci_bus 0000:00: resource 7 [mem 0xc0000000-0xfebfffff window] [ 0.871764] pci_bus 0000:00: resource 8 [mem 0x180000000-0x1ffffffff window] [ 0.874237] NET: Registered protocol family 2 [ 0.876571] IP idents hash table entries: 131072 (order: 8, 1048576 bytes, vmalloc) [ 0.880951] tcp_listen_portaddr_hash hash table entries: 4096 (order: 5, 163840 bytes, vmalloc) [ 0.885036] TCP established hash table entries: 65536 (order: 7, 524288 bytes, vmalloc) [ 0.890992] TCP bind hash table entries: 65536 (order: 9, 2097152 bytes, vmalloc) [ 0.894343] TCP: Hash tables configured (established 65536 bind 65536) [ 0.896859] MPTCP token hash table entries: 8192 (order: 6, 393216 bytes, vmalloc) [ 0.899679] UDP hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 0.902849] UDP-Lite hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 0.905412] NET: Registered protocol family 1 [ 0.910024] RPC: Registered named UNIX socket transport module. [ 0.911884] RPC: Registered udp transport module. [ 0.913413] RPC: Registered tcp transport module. [ 0.914937] RPC: Registered tcp NFSv4.1 backchannel transport module. [ 0.917418] NET: Registered protocol family 44 [ 0.918874] pci 0000:00:00.0: Limiting direct PCI/PCI transfers [ 0.920643] pci 0000:00:01.0: PIIX3: Enabling Passive Release [ 0.922591] pci 0000:00:01.0: Activating ISA DMA hang workarounds [ 0.924700] PCI: CLS 0 bytes, default 64 [ 0.926787] Unpacking initramfs... [ 2.655417] debug: unmapping init [mem 0xffff8d253cc54000-0xffff8d253ffbffff] [ 2.659700] PCI-DMA: Using software bounce buffering for IO (SWIOTLB) [ 2.661619] software IO TLB: mapped [mem 0x00000000a8000000-0x00000000ac000000] (64MB) [ 2.664935] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 3.481292] Initialise system trusted keyrings [ 3.483512] Key type blacklist registered [ 3.488773] workingset: timestamp_bits=36 max_order=20 bucket_order=0 [ 3.500860] zbud: loaded [ 3.504730] *** VALIDATE nfs *** [ 3.505819] *** VALIDATE nfs4 *** [ 3.507082] pstore: using deflate compression [ 3.512539] Platform Keyring initialized [ 3.695109] NET: Registered protocol family 38 [ 3.696494] Key type asymmetric registered [ 3.697672] Asymmetric key parser 'x509' registered [ 3.699911] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 247) [ 3.705835] io scheduler mq-deadline registered [ 3.709283] io scheduler kyber registered [ 3.711245] io scheduler bfq registered [ 3.712979] atomic64_test: passed for x86-64 platform with CX8 and with SSE [ 3.715880] shpchp: Standard Hot Plug PCI Controller Driver version: 0.4 [ 3.718789] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0 [ 3.721538] ACPI: Power Button [PWRF] [ 3.824562] ACPI: \_SB_.LNKB: Enabled at IRQ 10 [ 3.951730] ACPI: \_SB_.LNKA: Enabled at IRQ 11 [ 4.213640] ACPI: \_SB_.LNKC: Enabled at IRQ 11 [ 4.338207] ACPI: \_SB_.LNKD: Enabled at IRQ 10 [ 4.616363] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled [ 4.650371] 00:03: ttyS1 at I/O 0x2f8 (irq = 3, base_baud = 115200) is a 16550A [ 4.710633] 00:04: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A [ 4.725300] Non-volatile memory driver v1.3 [ 4.727143] Linux agpgart interface v0.103 [ 4.776420] virtio_blk virtio1: [vda] 133640 512-byte logical blocks (68.4 MB/65.3 MiB) [ 4.779963] vda: detected capacity change from 0 to 68423680 [ 4.797827] virtio_blk virtio2: [vdb] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) [ 4.811700] vdb: detected capacity change from 0 to 1073741824 [ 4.834978] virtio_blk virtio3: [vdc] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 4.837435] vdc: detected capacity change from 0 to 2621440000 [ 4.861856] virtio_blk virtio4: [vdd] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 4.864532] vdd: detected capacity change from 0 to 2621440000 [ 4.894540] virtio_blk virtio5: [vde] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 4.901506] vde: detected capacity change from 0 to 4294967296 [ 4.923501] virtio_blk virtio6: [vdf] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 4.926229] vdf: detected capacity change from 0 to 4294967296 [ 4.935832] libphy: Fixed MDIO Bus: probed [ 4.945327] usbcore: registered new interface driver usbserial_generic [ 4.947150] usbserial: USB Serial support registered for generic [ 4.950143] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12 [ 4.957417] serio: i8042 KBD port at 0x60,0x64 irq 1 [ 4.958867] serio: i8042 AUX port at 0x60,0x64 irq 12 [ 4.964542] mousedev: PS/2 mouse device common for all mice [ 4.967298] rtc_cmos 00:05: RTC can wake from S4 [ 4.975555] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input1 [ 4.978126] rtc_cmos 00:05: registered as rtc0 [ 4.979559] rtc_cmos 00:05: alarms up to one day, y3k, 242 bytes nvram, hpet irqs [ 4.985452] intel_pstate: CPU model not supported [ 4.986475] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input4 [ 4.997979] hid: raw HID events driver (C) Jiri Kosina [ 4.999587] usbcore: registered new interface driver usbhid [ 5.001246] usbhid: USB HID core driver [ 5.003230] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input3 [ 5.009473] drop_monitor: Initializing network drop monitor service [ 5.009595] Initializing XFRM netlink socket [ 5.009937] NET: Registered protocol family 10 [ 5.011187] Segment Routing with IPv6 [ 5.024946] NET: Registered protocol family 17 [ 5.026491] mpls_gso: MPLS GSO support [ 5.031537] RAS: Correctable Errors collector initialized. [ 5.033370] AVX version of gcm_enc/dec engaged. [ 5.034806] AES CTR mode by8 optimization enabled [ 5.157622] sched_clock: Marking stable (5157605081, 0)->(6267133603, -1109528522) [ 5.164231] registered taskstats version 1 [ 5.170121] Loading compiled-in X.509 certificates [ 5.171697] zswap: loaded using pool lzo/zbud [ 5.200518] Key type big_key registered [ 5.218108] Key type encrypted registered [ 5.219527] ima: No TPM chip found, activating TPM-bypass! [ 5.222074] ima: Allocated hash algorithm: sha1 [ 5.223384] ima: No architecture policies found [ 5.224753] evm: Initialising EVM extended attributes: [ 5.227580] evm: security.selinux [ 5.228571] evm: security.ima [ 5.229426] evm: security.capability [ 5.234787] evm: HMAC attrs: 0x1 [ 5.238118] rtc_cmos 00:05: setting system clock to 2025-12-03 19:41:55 UTC (1764790915) [ 5.244959] debug: unmapping init [mem 0xffffffff9cc03000-0xffffffff9cdfffff] [ 5.250687] debug: unmapping init [mem 0xffffffff9b982000-0xffffffff9bc58fff] [ 5.262062] Write protecting the kernel read-only data: 28672k [ 5.264988] debug: unmapping init [mem 0xffffffff9a003000-0xffffffff9a1fffff] [ 5.267347] debug: unmapping init [mem 0xffffffff9a914000-0xffffffff9a9fffff] [ 5.331651] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 5.354930] systemd[1]: Detected virtualization kvm. [ 5.356721] systemd[1]: Detected architecture x86-64. [ 5.358146] systemd[1]: Running in initial RAM disk. Welcome to Rocky Linux 8.10 (Green Obsidian) dracut-049-233.git20240115.el8 (Initramfs)! [ 5.394378] systemd[1]: No hostname configured. [ 5.396480] systemd[1]: Set hostname to . [ 5.398285] random: systemd: uninitialized urandom read (16 bytes read) [ 5.400934] systemd[1]: Initializing machine ID from random generator. [ 5.497641] random: ln: uninitialized urandom read (6 bytes read) [ 5.682218] random: systemd: uninitialized urandom read (16 bytes read) [ 5.685308] systemd[1]: Listening on udev Control Socket. [ OK ] Listening on udev Control Socket. [ 5.688967] systemd[1]: Reached target Initrd Root Device. [ OK ] Reached target Initrd Root Device. [ 5.695778] systemd[1]: Reached target Slices. [ OK ] Reached target Slices. [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Reached target Local Encrypted Volumes. [ OK ] Reached target Paths. [ OK ] Reached target Swap. [ OK ] Listening on udev Kernel Socket. [ OK ] Reached target Local File Systems. [ OK ] Listening on Journal Socket. Starting Create Volatile Files and Directories... [ OK ] Started Memstrack Anylazing Service. Starting Setup Virtual Console... [ OK ] Listening on Journal Socket (/dev/log). Starting Journal Service... [ OK ] Reached target Sockets. [ OK ] Reached target Timers. Starting Create list of required st…ce nodes for the current kernel... Starting Apply Kernel Variables... [ OK ] Started Create Volatile Files and Directories. [ OK ] Started Setup Virtual Console. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Started Apply Kernel Variables. Starting Create Static Device Nodes in /dev... Starting dracut cmdline hook... [ OK ] Started Create Static Device Nodes in /dev. [ OK ] Started Journal Service. [ OK ] Started dracut cmdline hook. Starting dracut pre-udev hook... [ 6.802891] device-mapper: uevent: version 1.0.3 [ 6.804729] device-mapper: ioctl: 4.46.0-ioctl (2022-02-22) initialised: dm-devel@redhat.com [ OK ] Started dracut pre-udev hook. Starting udev Kernel Device Manager... [ OK ] Started udev Kernel Device Manager. Starting dracut pre-trigger hook... [ OK ] Started dracut pre-trigger hook. Starting udev Coldplug all Devices... Mounting Kernel Configuration File System... [ OK ] Mounted Kernel Configuration File System. [ OK ] Started udev Coldplug all Devices. [ OK ] Reached target System Initialization. [ OK ] Reached target Basic System. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. Starting dracut initqueue hook... [ 8.288761] virtio_net virtio0 ens2: renamed from eth0 [ 8.494346] random: fast init done [ 8.844785] scsi host0: ata_piix [ 8.894927] scsi host1: ata_piix [ 8.898574] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc320 irq 14 [ 8.912141] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc328 irq 15 [ 13.745205] random: crng init done [ 13.746282] random: 7 urandom warning(s) missed due to ratelimiting [ 14.767552] dracut-initqueue[591]: RTNETLINK answers: File exists Starting nbd nbd0... [ OK ] Started nbd nbd0. [ OK ] Started dracut initqueue hook. Mounting /sysroot... [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. [ 16.395607] EXT4-fs (nbd0): mounted filesystem with ordered data mode. Opts: (null) [ OK ] Mounted /sysroot. [ OK ] Reached target Initrd Root File System. Starting Reload Configuration from the Real Root... [ OK ] Started Reload Configuration from the Real Root. [ OK ] Reached target Initrd File Systems. [ OK ] Reached target Initrd Default Target. Starting dracut pre-pivot and cleanup hook... [ OK ] Started dracut pre-pivot and cleanup hook. Starting Cleaning Up and Shutting Down Daemons... [ OK ] Stopped target Timers. Stopping Hardware RNG Entropy Gatherer Daemon... [ OK ] Stopped dracut pre-pivot and cleanup hook. [ OK ] Stopped target Initrd Default Target. [ OK ] Stopped target Initrd Root Device. [ OK ] Stopped target Remote File Systems. [ OK ] Stopped target Remote File Systems (Pre). [ OK ] Stopped dracut initqueue hook. [ OK ] Stopped Hardware RNG Entropy Gatherer Daemon. [ OK ] Stopped target Basic System. [ OK ] Stopped target Paths. [ OK ] Stopped target Sockets. [ OK ] Stopped target System Initialization. [ OK ] Stopped udev Coldplug all Devices. [ OK ] Stopped dracut pre-trigger hook. [ OK ] Stopped target Local Encrypted Volumes. [ OK ] Stopped Dispatch Password Requests to Console Directory Watch. [ OK ] Stopped Apply Kernel Variables. [ OK ] Stopped Create Volatile Files and Directories. [ OK ] Stopped target Swap. [ OK ] Stopped target Local File Systems. Stopping udev Kernel Device Manager... [ OK ] Stopped target Slices. [ OK ] Stopped udev Kernel Device Manager. [ OK ] Stopped dracut pre-udev hook. [ OK ] Stopped dracut cmdline hook. [ OK ] Stopped Create Static Device Nodes in /dev. [ OK ] Stopped Create list of required sta…vice nodes for the current kernel. [ OK ] Closed udev Kernel Socket. [ OK ] Closed udev Control Socket. Starting Cleanup udevd DB... [ OK ] Started Cleaning Up and Shutting Down Daemons. [ OK ] Started Cleanup udevd DB. [ OK ] Reached target Switch Root. Starting Switch Root... [ 18.294498] printk: systemd: 26 output lines suppressed due to ratelimiting [ 18.901214] SELinux: Disabled at runtime. [ 19.036285] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 19.050321] systemd[1]: Detected virtualization kvm. [ 19.052264] systemd[1]: Detected architecture x86-64. Welcome to Rocky Linux 8.10 (Green Obsidian)! [ 20.172376] systemd[1]: initrd-switch-root.service: Succeeded. [ 20.179043] systemd[1]: Stopped Switch Root. [ OK ] Stopped Switch Root. [ 20.187916] systemd[1]: systemd-journald.service: Service has no hold-off time (RestartSec=0), scheduling restart. [ 20.197921] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. [ 20.204383] systemd[1]: Stopped Journal Service. [ OK ] Stopped Journal Service. [ 20.230402] systemd[1]: Starting Journal Service... Starting Journal Service... [ 20.255656] systemd[1]: Created slice system-getty.slice. [ OK ] Created slice system-getty.slice. [ OK ] Listening on udev Kernel Socket. [ OK ] Listening on Process Core Dump Socket. Starting Apply Kernel Variables... [ OK ] Created slice system-sshd\x2dkeygen.slice. [ OK ] Listening on initctl Compatibility Named Pipe. Starting Create list of required st…ce nodes for the current kernel... [ OK ] Created slice User and Session Slice. [ OK ] Reached target Slices. [FAILED] Failed to set up automount Arbitrar…rmats File System Automount Point. See 'systemctl status proc-sys-fs-binfmt_misc.automount' for details. Mounting POSIX Message Queue File System... [ OK ] Listening on RPCbind Server Activation Socket. [ OK ] Stopped target Switch Root. [ OK ] Stopped target Initrd Root File System. [ OK ] Started Dispatch Password Requests to Console Directory Watch. Starting Remount Root and Kernel File Systems... [ OK ] Stopped target Initrd File Systems. [ OK ] Reached target rpc_pipefs.target. [ OK ] Created slice system-serial\x2dgetty.slice. [ OK ] Started Forward Password Requests to Wall Directory Watch. [ OK ] Reached target Paths. [ OK ] Reached target Local Encrypted Volumes. [ OK ] Listening on udev Control Socket. Starting udev Coldplug all Devices... Activating swap /dev/disk/by-label/SWAP... [ OK ] Reached target RPC Port Mapper. Mounting Huge Pages File System... Mounting Kernel Debug File System... [ 20.704204] Adding 1048572k swap on /dev/vdb. Priority:-2 extents:1 across:1048572k FS [ OK ] Started Journal Service. [ OK ] Started Apply Kernel Variables. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Mounted POSIX Message Queue File System. [FAILED] Failed to start Remount Root and Kernel File Systems. See 'systemctl status systemd-remount-fs.service' for details. [ OK ] Activated swap /dev/disk/by-label/SWAP. [ OK ] Mounted Huge Pages File System. [ OK ] Mounted Kernel Debug File System. [ OK ] Reached target Swap. Starting Configure read-only root support... Starting Create Static Device Nodes in /dev... Starting Flush Journal to Persistent Storage... [ OK ] Started Flush Journal to Persistent Storage. [ OK ] Started udev Coldplug all Devices. [ OK ] Started Create Static Device Nodes in /dev. [ OK ] Reached target Local File Systems (Pre). Mounting /mnt... Mounting /home/green/git/lustre-release... Starting udev Kernel Device Manager... [ OK ] Mounted /mnt. [ 21.393739] squashfs: version 4.0 (2009/01/31) Phillip Lougher [ OK ] Mounted /home/green/git/lustre-release. [ OK ] Started udev Kernel Device Manager. [ 22.355908] piix4_smbus 0000:00:01.3: SMBus Host Controller at 0x700, revision 0 [ 22.424062] input: PC Speaker as /devices/platform/pcspkr/input/input5 [ 23.551842] RAPL PMU: API unit is 2^-32 Joules, 0 fixed counters, 10737418240 ms ovfl timer [ 23.694452] EDAC sbridge: Ver: 1.1.2 [ 26.552304] Key type dns_resolver registered [ 27.011311] NFS: Registering the id_resolver key type [ 27.013077] Key type id_resolver registered [ 27.014535] Key type id_legacy registered [* ] A start job is running for Configur…-only root support (7s / no limit) [ OK ] Started Configure read-only root support. Starting Load/Save Random Seed... [ OK ] Reached target Local File Systems. Starting Mark the need to relabel after reboot... Starting Rebuild Dynamic Linker Cache... Starting Create Volatile Files and Directories... [ OK ] Started Load/Save Random Seed. [ OK ] Started Mark the need to relabel after reboot. [ OK ] Started Create Volatile Files and Directories. Starting RPC Bind... Starting Update UTMP about System Boot/Shutdown... [ OK ] Started Update UTMP about System Boot/Shutdown. [ OK ] Started RPC Bind. [ OK ] Started Rebuild Dynamic Linker Cache. Starting Update is Completed... [ OK ] Started Update is Completed. [ OK ] Reached target System Initialization. [ OK ] Started Daily Cleanup of Temporary Directories. [ OK ] Listening on D-Bus System Message Bus Socket. [ OK ] Reached target Sockets. [ OK ] Reached target Basic System. Starting Login Service... [ OK ] Reached target sshd-keygen.target. [ OK ] Started daily update of the root trust anchor for DNSSEC. Starting Restore /run/initramfs on shutdown... [ OK ] Started dnf makecache --timer. [ OK ] Reached target Timers. [ OK ] Started D-Bus System Message Bus. Starting Network Manager... [ OK ] Started Hardware RNG Entropy Gatherer Daemon. [ OK ] Started irqbalance daemon. [ OK ] Started Restore /run/initramfs on shutdown. [ OK ] Started Login Service. [ OK ] Started Network Manager. [ OK ] Reached target Network. Starting Dynamic System Tuning Daemon... Starting GSSAPI Proxy Daemon... Starting OpenSSH server daemon... Starting Network Manager Wait Online... Starting Hostname Service... [ OK ] Started GSSAPI Proxy Daemon. [ OK ] Reached target NFS client services. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Starting Permit User Sessions... [ OK ] Started Permit User Sessions. [ OK ] Started Command Scheduler. [ OK ] Started Serial Getty on ttyS1. [ OK ] Started Getty on tty1. [ OK ] Started Serial Getty on ttyS0. [ OK ] Reached target Login Prompts. [ OK ] Started OpenSSH server daemon. [ OK ] Started Hostname Service. Starting Network Manager Script Dispatcher Service... [ OK ] Started Network Manager Script Dispatcher Service. [ OK ] Started Network Manager Wait Online. [ OK ] Reached target Network is Online. Starting Crash recovery kernel arming... Starting Notify NFS peers of a restart... Starting System Logging Service... [ OK ] Started Notify NFS peers of a restart. [ OK ] Started System Logging Service. Rocky Linux 8.10 (Green Obsidian) Kernel 4.18.0rh8.10-debug on an x86_64 oleg224-server login: [ 52.524524] libcfs: loading out-of-tree module taints kernel. [ 52.536844] Key type ._llcrypt registered [ 52.538562] Key type .llcrypt registered [ 52.579509] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing set_hostid [ 59.704608] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing load_modules_local [ 60.279529] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 60.286069] alg: No test for adler32 (adler32-zlib) [ 61.259402] Lustre: Lustre: Build Version: 2.16.59_23_g5b5e124 [ 61.535066] LNet: Added LNI 192.168.202.124@tcp [8/256/0/180] [ 63.151143] Key type lgssc registered [ 63.677425] Lustre: Echo OBD driver; http://www.lustre.org/ [ 70.095409] ZFS: Loaded module v2.3.2-1, ZFS pool version 5000, ZFS filesystem version 5 [ 70.465240] LDISKFS-fs (vdc): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 73.357725] LDISKFS-fs (vdd): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 76.072755] LDISKFS-fs (vde): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 78.792966] LDISKFS-fs (vdf): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 84.270729] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing load_modules_local [ 89.323338] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 89.344865] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 89.351560] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 90.451044] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 90.465430] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 90.507427] Lustre: lustre-MDT0000: new disk, initializing [ 90.535902] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 90.546369] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 91.974328] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 97.725105] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 97.764271] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 97.799753] Lustre: 6488:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 97.814372] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 97.817522] Lustre: Skipped 1 previous similar message [ 97.864496] Lustre: lustre-MDT0001: new disk, initializing [ 97.891066] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 97.900527] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 97.905990] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 99.288856] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 101.615173] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 104.682173] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 104.705803] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 104.787380] Lustre: lustre-OST0000: new disk, initializing [ 104.788856] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 104.803308] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 104.868256] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 104.870676] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 104.902054] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 106.610661] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 111.875725] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 111.899678] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 111.934818] Lustre: lustre-OST0001: new disk, initializing [ 111.936419] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 111.953106] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 113.734661] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 120.382054] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 120.853187] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 120.859377] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 120.897062] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 125.618561] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 134.804615] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing check_logdir /tmp/testlogs/ [ 138.231359] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing yml_node [ 142.699465] Lustre: DEBUG MARKER: Client: 2.16.59.23 [ 144.854642] Lustre: DEBUG MARKER: MDS: 2.16.59.23 [ 146.869750] Lustre: DEBUG MARKER: OSS: 2.16.59.23 [ 148.217235] Lustre: DEBUG MARKER: -----============= acceptance-small: conf-sanity ============----- Wed Dec 3 14:44:18 EST 2025 [ 158.205779] Lustre: DEBUG MARKER: excepting tests: 102 106 115 32newtarball [ 158.870889] Lustre: DEBUG MARKER: skipping tests SLOW=no: 45 69 106 111 114 [ 171.999934] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 172.000654] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 172.004322] Lustre: Skipped 1 previous similar message [ 172.009167] Lustre: Skipped 3 previous similar messages [ 177.120063] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 177.123694] Lustre: Skipped 3 previous similar messages [ 177.216375] LustreError: 12191:0:(obd_class.h:479:obd_check_dev()) Device 28 not setup [ 177.243898] Lustre: server umount lustre-MDT0000 complete [ 178.750652] LustreError: 8395:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1764791089 with bad export cookie 8566939408257394038 [ 178.750834] LustreError: MGC192.168.202.124@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 178.758609] LustreError: 8395:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 182.241557] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 182.242076] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 182.246227] Lustre: Skipped 3 previous similar messages [ 182.250551] Lustre: Skipped 1 previous similar message [ 184.891962] LustreError: 12394:0:(obd_class.h:479:obd_check_dev()) Device 27 not setup [ 184.894832] LustreError: 12394:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 184.983865] Lustre: server umount lustre-MDT0001 complete [ 186.542405] LustreError: 12595:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 186.545049] LustreError: 12595:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 186.568930] Lustre: server umount lustre-OST0000 complete [ 188.228719] Lustre: server umount lustre-OST0001 complete [ 190.943393] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing set_hostid [ 195.023874] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing load_modules_local [ 199.751713] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 202.844401] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 205.764655] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 208.887976] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 212.258983] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 212.296795] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 212.417955] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 212.436102] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 212.484805] Lustre: lustre-MDT0000: new disk, initializing [ 212.556264] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 212.565096] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 214.245050] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing set_default_debug -1 all [ 220.602951] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 220.642686] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 220.710676] Lustre: 14974:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 220.748233] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 220.750758] Lustre: Skipped 1 previous similar message [ 220.822734] Lustre: lustre-MDT0001: new disk, initializing [ 220.920064] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 220.957536] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 220.963236] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 223.356984] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing set_default_debug -1 all [ 229.207397] Lustre: DEBUG MARKER: oleg224-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 232.218247] Lustre: DEBUG MARKER: oleg224-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 236.239534] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 236.280292] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 238.014784] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 238.019660] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 238.126515] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 238.339031] hrtimer: interrupt took 3194683 ns [ 240.481893] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing set_default_debug -1 all [ 246.192710] Lustre: DEBUG MARKER: oleg224-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 248.752186] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 248.897172] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 250.872719] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 250.973811] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 253.411589] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 253.411891] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 253.412648] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 253.412653] Lustre: Skipped 2 previous similar messages [ 253.415093] LustreError: Skipped 1 previous similar message [ 253.420385] Lustre: Skipped 1 previous similar message [ 258.532717] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 258.535599] Lustre: Skipped 1 previous similar message [ 266.207109] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 266.247161] LustreError: 17910:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 266.254337] LustreError: 17910:0:(obd_class.h:479:obd_check_dev()) Skipped 9 previous similar messages [ 266.365015] Lustre: server umount lustre-OST0000 complete [ 268.769939] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 268.773688] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 268.777324] Lustre: Skipped 1 previous similar message [ 268.783767] Lustre: Skipped 2 previous similar messages [ 274.645114] LustreError: 18113:0:(obd_class.h:479:obd_check_dev()) Device 22 not setup [ 274.647951] LustreError: 18113:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 274.727233] Lustre: server umount lustre-MDT0000 complete [ 277.020521] LustreError: 14965:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1764791187 with bad export cookie 8566939408257395557 [ 277.027240] LustreError: MGC192.168.202.124@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 277.032583] LustreError: 14965:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 277.316939] Lustre: server umount lustre-MDT0001 complete [ 282.777902] Lustre: DEBUG MARKER: == conf-sanity test 76a: set permanent params with lctl across mounts ========================================================== 14:46:31 (1764791191) [ 293.540777] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 294.533475] LustreError: 18840:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 294.575471] LustreError: 18840:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 5 previous similar messages [ 294.605429] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 294.611869] Lustre: Skipped 1 previous similar message [ 300.002218] LustreError: 18840:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 300.427182] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing set_default_debug -1 all [ 304.098444] LustreError: 18840:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 309.236594] LustreError: 18840:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 313.391898] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 321.098870] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing set_default_debug -1 all [ 331.509796] Lustre: DEBUG MARKER: oleg224-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 338.625531] Lustre: DEBUG MARKER: oleg224-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 348.192342] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 348.701662] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 348.708907] Lustre: Skipped 1 previous similar message [ 354.292330] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing set_default_debug -1 all [ 359.495247] Lustre: DEBUG MARKER: oleg224-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 368.012536] Lustre: Modifying parameter general.osc.*.max_dirty_mb=452 in log params [ 374.755747] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 374.761153] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 374.761656] Lustre: Skipped 1 previous similar message [ 374.775376] Lustre: Skipped 4 previous similar messages [ 379.432839] LustreError: 21148:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup [ 379.435094] LustreError: 21148:0:(obd_class.h:479:obd_check_dev()) Skipped 13 previous similar messages [ 379.504331] Lustre: server umount lustre-MDT0000 complete [ 379.884298] LustreError: 18836:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 379.909112] LustreError: 18836:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 382.789757] LustreError: 18821:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1764791293 with bad export cookie 8566939408257396439 [ 382.793527] LustreError: MGC192.168.202.124@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 382.808371] LustreError: 18821:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 383.206889] Lustre: server umount lustre-MDT0001 complete [ 397.120243] LustreError: 21552:0:(obd_class.h:479:obd_check_dev()) Device 2 not setup [ 397.125760] LustreError: 21552:0:(obd_class.h:479:obd_check_dev()) Skipped 11 previous similar messages [ 397.285389] Lustre: server umount lustre-OST0000 complete [ 411.814789] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing load_modules_local [ 423.468870] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 424.374982] LustreError: 22878:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 424.521754] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 431.022574] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing set_default_debug -1 all [ 440.817813] LustreError: 22878:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 440.853284] LustreError: 22878:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 444.367604] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 450.479806] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing set_default_debug -1 all [ 453.923653] Lustre: Setting parameter general.debug_raw_pointers=Y in log params [ 461.219361] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 467.043928] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing set_default_debug -1 all [ 474.875994] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 474.936910] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 475.072492] Lustre: lustre-OST0001: new disk, initializing [ 475.074272] Lustre: Skipped 1 previous similar message [ 475.077332] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 475.080101] Lustre: Skipped 1 previous similar message [ 476.150978] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 476.159974] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 476.239147] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 479.927630] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing set_default_debug -1 all [ 487.991246] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 490.968985] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 506.337799] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 506.342356] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 506.352785] Lustre: Skipped 1 previous similar message [ 506.358047] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 511.700890] LustreError: 26363:0:(obd_class.h:479:obd_check_dev()) Device 28 not setup [ 511.704177] LustreError: 26363:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 511.797131] Lustre: server umount lustre-MDT0000 complete [ 511.971533] LustreError: 22878:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 511.980462] LustreError: 22878:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 514.017209] LustreError: 22859:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1764791424 with bad export cookie 8566939408257397286 [ 514.018541] LustreError: MGC192.168.202.124@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 514.022213] LustreError: 22859:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 514.267680] Lustre: server umount lustre-MDT0001 complete [ 527.383148] Lustre: server umount lustre-OST0000 complete [ 540.647803] Lustre: server umount lustre-OST0001 complete [ 553.213643] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing load_modules_local [ 561.165726] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 561.642531] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 561.649357] Lustre: Skipped 3 previous similar messages [ 564.689766] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing set_default_debug -1 all [ 571.712069] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 575.297106] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing set_default_debug -1 all [ 577.437826] Lustre: 29257:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 582.989865] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 586.340670] LustreError: 29611:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 586.359534] LustreError: 29611:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 586.940855] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing set_default_debug -1 all [ 591.859394] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 595.463528] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing set_default_debug -1 all [ 603.303346] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 604.852455] Lustre: 31109:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 612.832020] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 612.834539] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 612.838535] Lustre: Skipped 3 previous similar messages [ 612.840539] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 612.842571] Lustre: Skipped 3 previous similar messages [ 617.666253] LustreError: 31312:0:(obd_class.h:479:obd_check_dev()) Device 10 not setup [ 617.669240] LustreError: 31312:0:(obd_class.h:479:obd_check_dev()) Skipped 25 previous similar messages [ 617.700109] Lustre: server umount lustre-MDT0000 complete [ 619.006066] LustreError: 28120:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1764791529 with bad export cookie 8566939408257398581 [ 619.009606] LustreError: MGC192.168.202.124@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 619.010026] LustreError: 28120:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 619.160058] Lustre: server umount lustre-MDT0001 complete [ 630.801935] Lustre: server umount lustre-OST0000 complete [ 642.680320] Lustre: server umount lustre-OST0001 complete [ 645.939966] Lustre: DEBUG MARKER: == conf-sanity test 76b: verify params log setup correctly ========================================================== 14:52:36 (1764791556) [ 655.535275] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing load_modules_local [ 659.038557] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 660.465335] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing set_default_debug -1 all [ 664.983729] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 668.334641] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing set_default_debug -1 all [ 670.989981] Lustre: 35010:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 676.884362] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 681.332937] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing set_default_debug -1 all [ 688.412717] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 692.966347] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing set_default_debug -1 all [ 698.480291] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 700.207977] Lustre: 36865:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 707.043750] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 707.049190] Lustre: Skipped 5 previous similar messages [ 712.929983] Lustre: server umount lustre-MDT0000 complete [ 714.418482] LustreError: 33871:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1764791624 with bad export cookie 8566939408257399687 [ 714.422099] LustreError: MGC192.168.202.124@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 714.422531] LustreError: 33871:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 714.578371] Lustre: server umount lustre-MDT0001 complete [ 726.532992] Lustre: server umount lustre-OST0000 complete [ 738.237602] Lustre: server umount lustre-OST0001 complete [ 741.578861] Lustre: DEBUG MARKER: == conf-sanity test 76c: verify changelog_mask is applied with lctl set_param -P ========================================================== 14:54:12 (1764791652) [ 747.411562] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing load_modules_local [ 751.386150] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 751.578218] LustreError: 38992:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 751.587901] LustreError: 38992:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 8 previous similar messages [ 753.045539] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing set_default_debug -1 all [ 756.041809] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 757.560593] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing set_default_debug -1 all [ 758.617409] Lustre: 40110:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 760.961940] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 763.141058] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing set_default_debug -1 all [ 766.225187] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 768.519347] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing set_default_debug -1 all [ 774.925770] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 784.635629] Lustre: Setting parameter general.mdd.*.changelog_mask=-CLOSE in log params [ 784.638891] Lustre: Skipped 1 previous similar message [ 790.121153] Lustre: lustre-MDT0000: Not available for connect from 192.168.202.24@tcp (stopping) [ 790.129513] Lustre: Skipped 12 previous similar messages [ 792.039447] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 792.048580] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 792.071277] Lustre: Skipped 4 previous similar messages [ 794.174400] LustreError: 42175:0:(obd_class.h:479:obd_check_dev()) Device 10 not setup [ 794.186843] LustreError: 42175:0:(obd_class.h:479:obd_check_dev()) Skipped 51 previous similar messages [ 794.369323] Lustre: server umount lustre-MDT0000 complete [ 798.790274] LustreError: 39727:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1764791709 with bad export cookie 8566939408257400793 [ 798.797169] LustreError: MGC192.168.202.124@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 798.873225] LustreError: 39727:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 805.465856] Lustre: server umount lustre-MDT0001 complete [ 815.018304] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 824.736657] LustreError: 3630:0:(client.c:1380:ptlrpc_import_delay_req()) @@@ invalidate in flight req@ffff8d25bde75c00 x1850517457559424/t0(0) o250->MGC192.168.202.124@tcp@0@lo:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl Rpc:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 825.381563] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 825.384235] Lustre: Skipped 11 previous similar messages [ 831.219751] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing set_default_debug -1 all [ 834.548310] Lustre: lustre-MDT0000-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 842.291105] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 847.117666] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing set_default_debug -1 all [ 848.387255] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 848.402785] Lustre: Skipped 1 previous similar message [ 856.198410] Lustre: DEBUG MARKER: oleg224-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 859.340501] Lustre: DEBUG MARKER: mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec [ 864.345328] Lustre: DEBUG MARKER: oleg224-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 867.274066] Lustre: DEBUG MARKER: mdc.lustre-MDT0001-mdc-*.mds_server_uuid in FULL state after 0 sec [ 878.563975] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 880.125115] Lustre: server umount lustre-MDT0000 complete [ 883.521150] LustreError: 39727:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1764791793 with bad export cookie 8566939408257401983 [ 883.531186] LustreError: MGC192.168.202.124@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 883.538285] LustreError: 39727:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 883.999280] Lustre: server umount lustre-MDT0001 complete [ 897.874771] Lustre: server umount lustre-OST0000 complete [ 900.408120] Lustre: 3632:0:(client.c:2468:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1764791794/real 1764791794] req@ffff8d25bded5880 x1850517457596416/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1764791810 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 901.217371] Lustre: server umount lustre-OST0001 complete [ 909.111581] Lustre: DEBUG MARKER: == conf-sanity test 76d: verify llite.*.xattr_cache can be set by 'lctl set_param -P' correctly ========================================================== 14:56:58 (1764791818) [ 924.831774] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing load_modules_local [ 933.655235] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 937.786673] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing set_default_debug -1 all [ 945.373330] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 949.128726] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing set_default_debug -1 all [ 951.783378] Lustre: 47456:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 951.793742] Lustre: 47456:0:(mgs_llog.c:1346:mgs_modify_param()) Skipped 1 previous similar message [ 957.491881] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 962.442933] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing set_default_debug -1 all [ 969.860927] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 975.851974] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing set_default_debug -1 all [ 982.756257] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 991.445280] Lustre: Setting parameter general.llite.*.xattr_cache=0 in log params [ 1004.040365] Lustre: server umount lustre-MDT0000 complete [ 1007.082665] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1007.084899] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1007.101991] Lustre: Skipped 8 previous similar messages [ 1007.800634] LustreError: 46317:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1764791918 with bad export cookie 8566939408257402767 [ 1007.806956] LustreError: MGC192.168.202.124@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1007.817720] LustreError: 46317:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1008.131360] Lustre: server umount lustre-MDT0001 complete [ 1021.537486] Lustre: server umount lustre-OST0000 complete [ 1035.600859] Lustre: server umount lustre-OST0001 complete [ 1048.777643] Lustre: DEBUG MARKER: == conf-sanity test 77: comma-separated MGS NIDs and failover node NIDs ========================================================== 14:59:17 (1764791957) [ 1052.034322] Lustre: DEBUG MARKER: SKIP: conf-sanity test_77 mixed loopback and real device not working [ 1054.583575] Lustre: DEBUG MARKER: == conf-sanity test 78: run resize2fs on MDT and OST filesystems ========================================================== 14:59:23 (1764791963) [ 1064.578535] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing set_hostid [ 1072.226886] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing load_modules_local [ 1080.381037] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1086.671907] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1095.282476] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1095.390652] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1095.797409] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1095.949097] Lustre: lustre-MDT0000: new disk, initializing [ 1096.109323] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1100.696503] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing set_default_debug -1 all [ 1109.098512] Lustre: DEBUG MARKER: oleg224-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1115.408562] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1115.474905] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1117.371878] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 1117.413862] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 1121.349949] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing set_default_debug -1 all [ 1128.702777] Lustre: DEBUG MARKER: oleg224-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1130.968432] Lustre: DEBUG MARKER: create test files [ 1160.775234] LustreError: 54158:0:(obd_class.h:479:obd_check_dev()) Device 12 not setup [ 1160.785984] LustreError: 54158:0:(obd_class.h:479:obd_check_dev()) Skipped 67 previous similar messages [ 1161.281390] Lustre: server umount lustre-OST0000 complete [ 1174.174338] Lustre: server umount lustre-MDT0000 complete [ 1182.263195] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing unload_modules_local [ 1184.817895] Key type lgssc unregistered [ 1185.053715] LNet: 54893:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1185.062691] LNetError: 54893:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1185.079286] LNet: Removed LNI 192.168.202.124@tcp [ 1185.871542] Key type .llcrypt unregistered [ 1185.874488] Key type ._llcrypt unregistered [ 1214.001089] Key type ._llcrypt registered [ 1214.011542] Key type .llcrypt registered [ 1214.192358] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing load_modules_local [ 1215.543560] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1215.752820] alg: No test for adler32 (adler32-zlib) [ 1216.902626] Lustre: Lustre: Build Version: 2.16.59_23_g5b5e124 [ 1217.127467] LNet: Added LNI 192.168.202.124@tcp [8/256/0/180] [ 1218.863192] Key type lgssc registered [ 1220.202854] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1228.460928] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1228.490564] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1230.155233] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1234.885868] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing set_default_debug -1 all [ 1243.104438] Lustre: DEBUG MARKER: oleg224-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1251.539947] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1252.031202] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1255.090420] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:102 to 0x240000400:129) [ 1255.234839] Lustre: 57496:0:(osd_internal.h:1360:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 515 < left 544, rollback = 9 [ 1255.250985] Lustre: 57496:0:(osd_handler.c:2081:osd_trans_dump_creds()) create: 32/128/0, destroy: 0/0/0 [ 1255.270607] Lustre: 57496:0:(osd_handler.c:2088:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 1255.287638] Lustre: 57496:0:(osd_handler.c:2098:osd_trans_dump_creds()) write: 2/13/1, punch: 0/0/0, quota 1/3/0 [ 1255.305292] Lustre: 57496:0:(osd_handler.c:2105:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 1255.315114] Lustre: 57496:0:(osd_handler.c:2112:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 1259.225551] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing set_default_debug -1 all [ 1266.870199] Lustre: DEBUG MARKER: oleg224-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1273.107862] Lustre: DEBUG MARKER: check files after expanding the MDT and OST filesystems [ 1279.474580] Lustre: DEBUG MARKER: create more files after expanding the MDT and OST filesystems [ 1291.234351] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1291.236061] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1291.276473] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1292.728171] LustreError: 58243:0:(obd_class.h:479:obd_check_dev()) Device 13 not setup [ 1292.993536] Lustre: server umount lustre-OST0000 complete [ 1297.185766] LustreError: 58445:0:(obd_class.h:479:obd_check_dev()) Device 8 not setup [ 1297.263245] LustreError: 58445:0:(obd_class.h:479:obd_check_dev()) Skipped 1 previous similar message [ 1297.517504] Lustre: server umount lustre-MDT0000 complete [ 1308.113686] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing unload_modules_local [ 1310.841978] Key type lgssc unregistered [ 1311.110265] LNet: 58978:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1311.116525] LNetError: 58978:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1311.133335] LNet: Removed LNI 192.168.202.124@tcp [ 1311.941175] Key type .llcrypt unregistered [ 1311.942986] Key type ._llcrypt unregistered [ 1341.322306] Key type ._llcrypt registered [ 1341.324057] Key type .llcrypt registered [ 1341.439286] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing load_modules_local [ 1342.358671] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1342.406744] alg: No test for adler32 (adler32-zlib) [ 1343.445713] Lustre: Lustre: Build Version: 2.16.59_23_g5b5e124 [ 1343.630135] LNet: Added LNI 192.168.202.124@tcp [8/256/0/180] [ 1345.280175] Key type lgssc registered [ 1346.361891] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1353.247746] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1353.267432] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1354.581342] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1357.527119] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing set_default_debug -1 all [ 1362.099557] Lustre: DEBUG MARKER: oleg224-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1367.850727] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1368.090053] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1372.512714] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing set_default_debug -1 all [ 1377.625335] Lustre: DEBUG MARKER: oleg224-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1379.376327] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:140 to 0x240000400:161) [ 1379.483708] Lustre: 61666:0:(osd_internal.h:1360:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 515 < left 544, rollback = 9 [ 1379.506682] Lustre: 61666:0:(osd_handler.c:2081:osd_trans_dump_creds()) create: 32/128/0, destroy: 0/0/0 [ 1379.523589] Lustre: 61666:0:(osd_handler.c:2088:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 1379.529196] Lustre: 61666:0:(osd_handler.c:2098:osd_trans_dump_creds()) write: 2/13/1, punch: 0/0/0, quota 1/3/0 [ 1379.539340] Lustre: 61666:0:(osd_handler.c:2105:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 1379.546269] Lustre: 61666:0:(osd_handler.c:2112:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 1388.471672] Lustre: DEBUG MARKER: check files after shrinking the MDT and OST filesystems [ 1399.783746] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1399.783866] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1399.801352] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1402.738200] LustreError: 62360:0:(obd_class.h:479:obd_check_dev()) Device 13 not setup [ 1403.120261] Lustre: server umount lustre-OST0000 complete [ 1407.716467] LustreError: 62562:0:(obd_class.h:479:obd_check_dev()) Device 8 not setup [ 1407.719434] LustreError: 62562:0:(obd_class.h:479:obd_check_dev()) Skipped 1 previous similar message [ 1408.153391] Lustre: server umount lustre-MDT0000 complete [ 1426.046497] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing unload_modules_local [ 1429.888872] Key type lgssc unregistered [ 1430.192344] LNet: 63117:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1430.207887] LNetError: 63117:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1430.248320] LNet: Removed LNI 192.168.202.124@tcp [ 1431.223753] Key type .llcrypt unregistered [ 1431.233139] Key type ._llcrypt unregistered [ 1449.946174] Key type ._llcrypt registered [ 1449.947952] Key type .llcrypt registered [ 1450.092950] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing set_hostid [ 1466.751499] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing load_modules_local [ 1468.567293] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1468.587411] alg: No test for adler32 (adler32-zlib) [ 1469.701861] Lustre: Lustre: Build Version: 2.16.59_23_g5b5e124 [ 1469.968135] LNet: Added LNI 192.168.202.124@tcp [8/256/0/180] [ 1471.679240] Key type lgssc registered [ 1472.851668] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1482.256890] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1488.893659] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1494.895067] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1501.413901] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1508.054657] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1508.144623] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1508.167741] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1509.491750] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 1509.524207] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1509.626251] Lustre: lustre-MDT0000: new disk, initializing [ 1509.738125] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1509.766362] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1513.466974] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing set_default_debug -1 all [ 1524.205490] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1524.301765] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1524.419816] Lustre: 66244:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 1524.494042] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 1524.496663] Lustre: Skipped 1 previous similar message [ 1524.596613] Lustre: lustre-MDT0001: new disk, initializing [ 1524.683178] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 1524.726623] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 1524.740174] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 1528.889395] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing set_default_debug -1 all [ 1536.971779] Lustre: DEBUG MARKER: oleg224-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1541.106477] Lustre: DEBUG MARKER: oleg224-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 1547.384169] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1547.440155] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1547.629390] Lustre: lustre-OST0000: new disk, initializing [ 1547.637293] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1547.723859] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1552.529950] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing set_default_debug -1 all [ 1555.007294] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 1555.019394] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 1555.098227] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 1558.610598] Lustre: DEBUG MARKER: oleg224-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1562.248906] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 1562.456721] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 1565.829246] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 1566.003809] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 1570.793733] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1570.794286] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1570.797363] LustreError: Skipped 1 previous similar message [ 1570.802442] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1570.820991] Lustre: Skipped 1 previous similar message [ 1573.952136] LustreError: 69184:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 1574.116819] Lustre: server umount lustre-OST0000 complete [ 1581.026188] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1581.029023] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1581.035605] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1581.035615] Lustre: Skipped 1 previous similar message [ 1581.043093] Lustre: Skipped 1 previous similar message [ 1583.753542] LustreError: 69385:0:(obd_class.h:479:obd_check_dev()) Device 22 not setup [ 1583.777538] LustreError: 69385:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 1583.941164] Lustre: server umount lustre-MDT0000 complete [ 1587.220475] LustreError: 66237:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1764792497 with bad export cookie 16435533322016895390 [ 1587.225915] LustreError: MGC192.168.202.124@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1587.226890] LustreError: 66237:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1587.412362] LustreError: 69587:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 1587.423226] LustreError: 69587:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 1587.727262] Lustre: server umount lustre-MDT0001 complete [ 1595.274434] Lustre: DEBUG MARKER: == conf-sanity test 79: format MDT/OST without mgs option (should return errors) ========================================================== 15:08:25 (1764792505) [ 1604.161938] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing load_modules_local [ 1617.613855] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1647.553971] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing set_hostid [ 1659.331118] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing load_modules_local [ 1667.025988] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1672.939772] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1678.459019] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1684.227772] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1690.417110] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1690.469738] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1690.637083] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 1690.657760] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1690.736989] Lustre: lustre-MDT0000: new disk, initializing [ 1690.808894] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1690.836553] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1694.008935] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing set_default_debug -1 all [ 1702.705749] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1702.760344] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1702.817985] Lustre: 73828:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 1702.860226] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 1702.863460] Lustre: Skipped 1 previous similar message [ 1702.932458] Lustre: lustre-MDT0001: new disk, initializing [ 1703.031120] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 1703.063499] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 1703.069283] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 1706.623954] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing set_default_debug -1 all [ 1713.308585] Lustre: DEBUG MARKER: oleg224-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1717.013794] Lustre: DEBUG MARKER: oleg224-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 1723.545354] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1723.605865] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1723.760429] Lustre: lustre-OST0000: new disk, initializing [ 1723.763573] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1723.840537] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1729.607456] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing set_default_debug -1 all [ 1734.240366] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 1734.249108] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 1734.327495] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 1737.144211] Lustre: DEBUG MARKER: oleg224-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1740.903682] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 1741.125702] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 1744.701767] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 1744.911753] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 1749.988646] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1750.006274] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1750.025468] Lustre: Skipped 1 previous similar message [ 1750.029403] Lustre: Skipped 1 previous similar message [ 1750.054210] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 1755.115865] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1760.229556] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1760.243387] Lustre: Skipped 2 previous similar messages [ 1761.248772] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 1761.340383] LustreError: 76768:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 1761.343389] LustreError: 76768:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 1761.527603] Lustre: server umount lustre-OST0000 complete [ 1769.953175] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1769.966593] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1770.002135] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1771.766374] LustreError: 76973:0:(obd_class.h:479:obd_check_dev()) Device 22 not setup [ 1771.780614] LustreError: 76973:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 1771.892225] Lustre: server umount lustre-MDT0000 complete [ 1775.587468] LustreError: 73841:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1775.609205] LustreError: 73841:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 4 previous similar messages [ 1775.683739] LustreError: 76974:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1764792685 with bad export cookie 16435533322016896272 [ 1775.687857] LustreError: MGC192.168.202.124@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1775.712377] LustreError: 76974:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 1776.140787] Lustre: server umount lustre-MDT0001 complete [ 1784.180467] Lustre: DEBUG MARKER: == conf-sanity test 80: mgc import reconnect race ======== 15:11:33 (1764792693) [ 1790.872424] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1791.294922] LustreError: 77847:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1791.365547] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1795.238865] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing set_default_debug -1 all [ 1796.579795] LustreError: 77847:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1801.713550] LustreError: 77847:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1804.445601] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1809.807451] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing set_default_debug -1 all [ 1816.706319] Lustre: DEBUG MARKER: oleg224-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1821.178988] Lustre: DEBUG MARKER: oleg224-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 1827.763429] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1828.168809] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1828.175480] Lustre: Skipped 1 previous similar message [ 1834.337537] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing set_default_debug -1 all [ 1840.730468] Lustre: DEBUG MARKER: oleg224-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1844.695821] Lustre: 80188:0:(genops.c:1791:obd_export_evict_by_uuid()) MGS: evicting a20c9b25-7c39-472b-a01d-77661868aa60 at adminstrative request [ 1846.758560] LustreError: MGC192.168.202.124@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1846.775979] Lustre: *** cfs_fail_loc=906, val=2147483648*** [ 1857.000116] Lustre: MGS: Client a20c9b25-7c39-472b-a01d-77661868aa60 (at 0@lo) reconnecting [ 1857.010700] Lustre: Evicted from MGS (at 0@lo) after server handle changed from 0xe416be7a28e9b882 to 0xe416be7a28e9bb22 [ 1857.028508] Lustre: MGC192.168.202.124@tcp: Connection restored to 0@lo (at 0@lo) [ 1881.847120] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1881.942899] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1882.092760] Lustre: lustre-OST0001: new disk, initializing [ 1882.107228] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 1887.501245] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing set_default_debug -1 all [ 1888.859917] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 1888.876529] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 1889.001697] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 1894.213585] Lustre: DEBUG MARKER: oleg224-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid [ 1899.490248] LustreError: lustre-OST0001-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 1899.499626] Lustre: lustre-OST0001-osc-MDT0000: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1899.501371] LustreError: Skipped 1 previous similar message [ 1899.507994] Lustre: Skipped 1 previous similar message [ 1899.513996] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 1899.523447] Lustre: Skipped 3 previous similar messages [ 1902.712133] LustreError: 81323:0:(obd_class.h:479:obd_check_dev()) Device 25 not setup [ 1902.717689] LustreError: 81323:0:(obd_class.h:479:obd_check_dev()) Skipped 13 previous similar messages [ 1902.767087] Lustre: server umount lustre-OST0001 complete [ 1904.616350] LustreError: 79446:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1904.627462] LustreError: 79446:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 1908.194349] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1908.199573] LustreError: Skipped 1 previous similar message [ 1908.202424] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1908.213609] Lustre: Skipped 1 previous similar message [ 1911.816877] Lustre: server umount lustre-OST0000 complete [ 1921.716993] LustreError: 81726:0:(obd_class.h:479:obd_check_dev()) Device 28 not setup [ 1921.727380] LustreError: 81726:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 1921.830046] Lustre: server umount lustre-MDT0000 complete [ 1925.102962] LustreError: 77842:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1925.122705] LustreError: 77842:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 1925.611937] LustreError: 77827:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1764792835 with bad export cookie 16435533322016897826 [ 1925.618514] LustreError: MGC192.168.202.124@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1925.619585] LustreError: 77827:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1926.002742] Lustre: server umount lustre-MDT0001 complete [ 1934.134811] Lustre: DEBUG MARKER: == conf-sanity test 81: sparse OST indexing ============== 15:14:03 (1764792843) [ 1935.655809] Lustre: DEBUG MARKER: SKIP: conf-sanity test_81 needs >= 3 OSTs [ 1937.568541] Lustre: DEBUG MARKER: == conf-sanity test 82a: specify OSTs for file (succeed) or directory (succeed) ========================================================== 15:14:06 (1764792846) [ 1939.375524] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82a needs >= 3 OSTs [ 1941.737600] Lustre: DEBUG MARKER: == conf-sanity test 82b: specify OSTs for file with --pool and --ost-list options ========================================================== 15:14:10 (1764792850) [ 1943.686125] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82b needs >= 4 OSTs [ 1945.495583] Lustre: DEBUG MARKER: == conf-sanity test 83: ENOSPACE on OST doesn't cause message VFS: Busy inodes after unmount ... ========================================================== 15:14:14 (1764792854) [ 1947.108732] Lustre: DEBUG MARKER: mount the OST /dev/mapper/ost1_flakey as a ldiskfs filesystem [ 1953.366763] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1954.508350] loop: module loaded [ 1954.548084] loop0: detected capacity change from 0 to 4294967296 [ 1954.611747] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 1956.497528] Lustre: DEBUG MARKER: run llverfs in partial mode on the OST ldiskfs /mnt/lustre-ost1 [ 1960.856654] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing run_llverfs /mnt/lustre-ost1 -vpl no [ 1965.961702] Lustre: DEBUG MARKER: unmount the OST /dev/mapper/ost1_flakey [ 1974.941521] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1978.654805] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1981.837336] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1985.383264] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1993.093568] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1993.185482] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1993.213421] LustreError: 84505:0:(obd_config.c:783:class_setup()) setup lustre-OST0000-osd failed (-28) [ 1993.219525] LustreError: 84505:0:(obd_mount.c:193:lustre_start_simple()) lustre-OST0000-osd setup error -28 [ 1993.229344] LustreError: 84505:0:(tgt_mount.c:2447:server_fill_super()) Unable to start osd on /dev/mapper/ost1_flakey: -28 [ 1993.241373] LustreError: 84505:0:(super25.c:199:lustre_fill_super()) llite: Unable to mount : rc = -28 [ 2016.409592] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing set_hostid [ 2027.187871] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing load_modules_local [ 2036.727351] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2042.563582] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2048.112345] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2054.589518] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2060.868518] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2060.931788] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2061.121441] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2061.164069] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2061.272413] Lustre: lustre-MDT0000: new disk, initializing [ 2061.347755] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2061.357985] Lustre: Skipped 1 previous similar message [ 2061.400644] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2065.362170] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing set_default_debug -1 all [ 2074.307196] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2074.373865] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2074.481797] Lustre: 87348:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2074.738292] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2078.004121] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing set_default_debug -1 all [ 2085.408217] Lustre: DEBUG MARKER: oleg224-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2089.038992] Lustre: DEBUG MARKER: oleg224-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2095.391594] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2095.489661] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2100.306394] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing set_default_debug -1 all [ 2104.396737] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2104.547356] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2107.374487] Lustre: DEBUG MARKER: oleg224-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2111.092596] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2111.383173] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2116.051527] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2116.403849] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2120.164429] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2120.184824] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2120.189719] LustreError: Skipped 2 previous similar messages [ 2120.214918] Lustre: Skipped 2 previous similar messages [ 2120.236272] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2120.251118] Lustre: Skipped 8 previous similar messages [ 2124.500753] LustreError: 90290:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 2124.503505] LustreError: 90290:0:(obd_class.h:479:obd_check_dev()) Skipped 17 previous similar messages [ 2124.778979] Lustre: server umount lustre-OST0000 complete [ 2135.341858] Lustre: server umount lustre-MDT0000 complete [ 2135.529713] LustreError: 90107:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2135.583502] LustreError: 90107:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 2139.322127] LustreError: 87340:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1764793049 with bad export cookie 16435533322016898421 [ 2139.332879] LustreError: MGC192.168.202.124@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2139.348023] LustreError: 87340:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2139.860282] Lustre: server umount lustre-MDT0001 complete [ 2150.970532] Lustre: DEBUG MARKER: == conf-sanity test 84: check recovery_hard_time ========= 15:17:40 (1764793060) [ 2157.331182] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2161.175375] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing set_default_debug -1 all [ 2169.091357] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2174.229436] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing set_default_debug -1 all [ 2180.084648] Lustre: DEBUG MARKER: oleg224-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2184.292723] Lustre: DEBUG MARKER: oleg224-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2192.212852] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2192.517618] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2192.526746] Lustre: Skipped 4 previous similar messages [ 2197.600697] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing set_default_debug -1 all [ 2203.416228] Lustre: DEBUG MARKER: oleg224-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2207.642087] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2207.883575] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2212.608772] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2212.822839] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2221.665257] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2221.738507] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2221.861884] Lustre: lustre-OST0001: new disk, initializing [ 2221.863509] Lustre: Skipped 2 previous similar messages [ 2221.870122] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 2221.872649] Lustre: Skipped 3 previous similar messages [ 2230.943865] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 2230.966076] Lustre: Skipped 2 previous similar messages [ 2230.968409] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 2231.121361] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 2231.656374] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing set_default_debug -1 all [ 2241.000847] Lustre: DEBUG MARKER: oleg224-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid [ 2244.730279] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid 50 [ 2244.925480] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2248.721733] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [ 2248.973783] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2263.545231] Lustre: DEBUG MARKER: mds1 REPLAY BARRIER on lustre-MDT0000 [ 2288.522042] Lustre: Failing over lustre-MDT0000 [ 2289.653866] LustreError: 96272:0:(obd_class.h:479:obd_check_dev()) Device 28 not setup [ 2289.691954] LustreError: 96272:0:(obd_class.h:479:obd_check_dev()) Skipped 17 previous similar messages [ 2289.912675] Lustre: server umount lustre-MDT0000 complete [ 2292.356908] LustreError: 92479:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 192.168.202.24@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2292.367170] LustreError: 92479:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 4 previous similar messages [ 2292.704119] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2292.711467] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2292.727188] Lustre: Skipped 6 previous similar messages [ 2309.090681] Lustre: 64458:0:(client.c:2468:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1764793202/real 1764793202] req@000000008f9fd576 x1850518934909312/t0(0) o400->MGC192.168.202.124@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1764793218 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 2309.179316] LustreError: MGC192.168.202.124@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2327.734241] LDISKFS-fs (dm-0): 4 truncates cleaned up [ 2327.736107] LDISKFS-fs (dm-0): recovery complete [ 2327.759730] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2335.714952] LustreError: 64455:0:(client.c:1380:ptlrpc_import_delay_req()) @@@ invalidate in flight req@00000000994490c3 x1850518934922112/t0(0) o250->MGC192.168.202.124@tcp@0@lo:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl Rpc:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 2336.123937] Lustre: lustre-MDT0000: in recovery but waiting for the first client to connect [ 2338.377735] Lustre: lustre-MDT0000: Will be in recovery for at least 1:00, or until 3 clients reconnect [ 2338.379842] Lustre: 91359:0:(ldlm_lib.c:2067:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 2340.589499] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing set_default_debug -1 all [ 2341.346427] Lustre: 91364:0:(ldlm_lib.c:2067:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 2341.353996] Lustre: 91364:0:(ldlm_lib.c:2067:extend_recovery_timer()) Skipped 1 previous similar message [ 2341.367696] Lustre: lustre-MDT0000-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 2341.470833] LustreError: 97023:0:(ldlm_lib.c:2679:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2341.794084] LustreError: 97023:0:(ldlm_lib.c:2679:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2342.131504] LustreError: 97023:0:(ldlm_lib.c:2679:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2342.143382] LustreError: 97023:0:(ldlm_lib.c:2679:replay_request_or_update()) Skipped 1 previous similar message [ 2342.496711] LustreError: 97023:0:(ldlm_lib.c:2679:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2342.499304] LustreError: 97023:0:(ldlm_lib.c:2679:replay_request_or_update()) Skipped 1 previous similar message [ 2342.520797] Lustre: 97023:0:(ldlm_lib.c:2067:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2342.532458] Lustre: 97023:0:(ldlm_lib.c:2067:extend_recovery_timer()) Skipped 7 previous similar messages [ 2343.188850] LustreError: 97023:0:(ldlm_lib.c:2679:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2343.197392] LustreError: 97023:0:(ldlm_lib.c:2679:replay_request_or_update()) Skipped 2 previous similar messages [ 2343.511161] LustreError: 97023:0:(ldlm_lib.c:2679:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2343.524900] LustreError: 97023:0:(ldlm_lib.c:2679:replay_request_or_update()) Skipped 2 previous similar messages [ 2344.839989] Lustre: 97023:0:(ldlm_lib.c:2067:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2344.858384] Lustre: 97023:0:(ldlm_lib.c:2067:extend_recovery_timer()) Skipped 6 previous similar messages [ 2345.193163] LustreError: 97023:0:(ldlm_lib.c:2679:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2345.206975] LustreError: 97023:0:(ldlm_lib.c:2679:replay_request_or_update()) Skipped 5 previous similar messages [ 2345.535103] LustreError: 97023:0:(ldlm_lib.c:2679:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2345.544216] LustreError: 97023:0:(ldlm_lib.c:2679:replay_request_or_update()) Skipped 5 previous similar messages [ 2349.040804] Lustre: 97023:0:(ldlm_lib.c:2067:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2349.055278] Lustre: 97023:0:(ldlm_lib.c:2067:extend_recovery_timer()) Skipped 12 previous similar messages [ 2349.377687] LustreError: 97023:0:(ldlm_lib.c:2679:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2349.386055] LustreError: 97023:0:(ldlm_lib.c:2679:replay_request_or_update()) Skipped 12 previous similar messages [ 2349.706642] LustreError: 97023:0:(ldlm_lib.c:2679:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2349.709395] LustreError: 97023:0:(ldlm_lib.c:2679:replay_request_or_update()) Skipped 12 previous similar messages [ 2357.224117] Lustre: 97023:0:(ldlm_lib.c:2067:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2357.231976] Lustre: 97023:0:(ldlm_lib.c:2067:extend_recovery_timer()) Skipped 25 previous similar messages [ 2357.554064] LustreError: 97023:0:(ldlm_lib.c:2679:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2357.560943] LustreError: 97023:0:(ldlm_lib.c:2679:replay_request_or_update()) Skipped 25 previous similar messages [ 2357.879195] LustreError: 97023:0:(ldlm_lib.c:2679:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2357.882706] LustreError: 97023:0:(ldlm_lib.c:2679:replay_request_or_update()) Skipped 25 previous similar messages [ 2373.520617] Lustre: 97023:0:(ldlm_lib.c:2067:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2373.526565] Lustre: 97023:0:(ldlm_lib.c:2067:extend_recovery_timer()) Skipped 51 previous similar messages [ 2373.842750] LustreError: 97023:0:(ldlm_lib.c:2679:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2373.847784] LustreError: 97023:0:(ldlm_lib.c:2679:replay_request_or_update()) Skipped 51 previous similar messages [ 2374.167255] LustreError: 97023:0:(ldlm_lib.c:2679:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2374.170585] LustreError: 97023:0:(ldlm_lib.c:2679:replay_request_or_update()) Skipped 51 previous similar messages [ 2398.223181] Lustre: 97023:0:(ldlm_lib.c:2375:target_recovery_overseer()) lustre-MDT0000 recovery is aborted by hard timeout [ 2398.227679] Lustre: 97023:0:(ldlm_lib.c:2385:target_recovery_overseer()) recovery is aborted, evict exports in recovery [ 2398.231681] Lustre: 97023:0:(genops.c:1618:class_disconnect_stale_exports()) lustre-MDT0000: disconnect stale client cbea657b-39ed-4276-b1ee-674bcdb38578@192.168.202.24@tcp [ 2398.237630] Lustre: lustre-MDT0000: disconnecting 1 stale clients [ 2398.240522] Lustre: 97023:0:(ldlm_lib.c:1897:abort_req_replay_queue()) @@@ aborted: req@00000000eb589d27 x1850518917942784/t0(8589934777) o101->cbea657b-39ed-4276-b1ee-674bcdb38578@192.168.202.24@tcp:449/0 lens 592/0 e 0 to 0 dl 1764793319 ref 1 fl Complete:/604/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:0 [ 2398.258994] Lustre: lustre-MDT0000-osd: cancel update llog [0x200000400:0x1:0x0] [ 2398.263431] Lustre: lustre-MDT0000: Denying connection for new client cbea657b-39ed-4276-b1ee-674bcdb38578 (at 192.168.202.24@tcp), waiting for 3 known clients (2 recovered, 0 in progress, and 1 evicted) already passed deadline 0:00 [ 2398.275615] Lustre: lustre-MDT0001-osp-MDT0000: cancel update llog [0x240000401:0x1:0x0] [ 2398.297714] Lustre: lustre-MDT0000-osp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 2398.300429] Lustre: Skipped 2 previous similar messages [ 2398.301236] Lustre: lustre-MDT0000: Recovery over after 1:00, of 3 clients 2 recovered and 1 was evicted. [ 2398.326529] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:79 to 0x280000401:577) [ 2398.326529] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:47 to 0x2c0000401:545) [ 2402.790225] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2402.795240] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2402.798047] Lustre: Skipped 3 previous similar messages [ 2404.482700] Lustre: server umount lustre-OST0000 complete [ 2407.910149] LustreError: 93108:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2407.917083] LustreError: 93108:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 63 previous similar messages [ 2420.191122] Lustre: lustre-OST0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2420.211113] LustreError: 97889:0:(obd_class.h:479:obd_check_dev()) Device 25 not setup [ 2420.213636] LustreError: 97889:0:(obd_class.h:479:obd_check_dev()) Skipped 11 previous similar messages [ 2420.282081] Lustre: server umount lustre-OST0001 complete [ 2427.811623] Lustre: server umount lustre-MDT0000 complete [ 2429.279154] LustreError: 91346:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1764793339 with bad export cookie 16435533322016949640 [ 2429.282875] LustreError: MGC192.168.202.124@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2429.284802] LustreError: 91346:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2429.455194] Lustre: server umount lustre-MDT0001 complete [ 2432.616137] Lustre: DEBUG MARKER: == conf-sanity test 85: osd_ost init: fail ea_fid_set ==== 15:22:23 (1764793343) [ 2435.475474] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2435.479290] Lustre: *** cfs_fail_loc=197, val=0*** [ 2450.911606] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 2456.031468] LustreError: 98984:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.202.124@tcp: failed processing log, type 4: rc = -110 [ 2461.168128] Lustre: *** cfs_fail_loc=197, val=0*** [ 2461.170194] Lustre: Skipped 65 previous similar messages [ 2481.695326] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2481.699859] Lustre: Skipped 2 previous similar messages [ 2484.203850] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing set_default_debug -1 all [ 2486.530364] Lustre: DEBUG MARKER: oleg224-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2487.456469] Lustre: server umount lustre-OST0000 complete [ 2491.174683] Lustre: DEBUG MARKER: == conf-sanity test 86: Replacing mkfs.lustre -G option == 15:23:21 (1764793401) [ 2505.655747] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2507.442103] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing set_default_debug -1 all [ 2511.050900] Lustre: DEBUG MARKER: oleg224-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2513.276735] LustreError: 100524:0:(osp_object.c:617:osp_attr_get()) lustre-MDT0001-osp-MDT0000: osp_attr_get update error [0x200000009:0x1:0x0]: rc = -5 [ 2513.281700] LustreError: 100524:0:(lod_sub_object.c:917:lod_sub_prep_llog()) lustre-MDT0000-mdtlov: can't get id from catalogs: rc = -5 [ 2513.285836] LustreError: 100524:0:(lod_dev.c:510:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 7, retries 0, failed: rc = -5 [ 2513.443616] Lustre: server umount lustre-MDT0000 complete [ 2519.800829] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2526.806432] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing set_hostid [ 2531.068722] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing load_modules_local [ 2534.892883] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2537.363873] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2539.810263] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2542.343118] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2545.047845] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2545.075930] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2545.183794] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2545.197892] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2545.236346] Lustre: lustre-MDT0000: new disk, initializing [ 2545.275787] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2546.848507] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing set_default_debug -1 all [ 2551.777872] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2551.806134] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2551.841582] Lustre: 104557:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2551.954803] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2553.393925] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing set_default_debug -1 all [ 2556.986172] Lustre: DEBUG MARKER: oleg224-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2558.484703] Lustre: DEBUG MARKER: oleg224-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2561.360480] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2561.399565] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2563.903098] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing set_default_debug -1 all [ 2567.711929] Lustre: DEBUG MARKER: oleg224-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2568.254254] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2569.571163] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2569.679411] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2571.430087] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2571.524260] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2573.281374] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2573.283589] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2573.287823] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2573.294507] Lustre: Skipped 6 previous similar messages [ 2573.297046] Lustre: Skipped 8 previous similar messages [ 2574.569528] Lustre: server umount lustre-OST0000 complete [ 2582.639294] Lustre: server umount lustre-MDT0000 complete [ 2583.521114] LustreError: 105354:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2583.529230] LustreError: 105354:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 9 previous similar messages [ 2584.489177] LustreError: 105338:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1764793494 with bad export cookie 16435533322016960273 [ 2584.493521] LustreError: MGC192.168.202.124@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2584.493879] LustreError: 105338:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 2584.737509] Lustre: server umount lustre-MDT0001 complete [ 2588.530269] Lustre: DEBUG MARKER: == conf-sanity test 87: check if MDT inode can hold EAs with N stripes properly ========================================================== 15:24:58 (1764793498) [ 2592.130068] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing unload_modules_local [ 2593.506650] Key type lgssc unregistered [ 2593.662314] LNet: 108623:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2593.667080] LNetError: 108623:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2593.673715] LNet: Removed LNI 192.168.202.124@tcp [ 2594.049591] Key type .llcrypt unregistered [ 2594.051543] Key type ._llcrypt unregistered [ 2602.340905] Key type ._llcrypt registered [ 2602.342659] Key type .llcrypt registered [ 2602.396514] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing set_hostid [ 2609.998761] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing load_modules_local [ 2610.522743] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2610.590593] alg: No test for adler32 (adler32-zlib) [ 2611.477545] Lustre: Lustre: Build Version: 2.16.59_23_g5b5e124 [ 2611.620873] LNet: Added LNI 192.168.202.124@tcp [8/256/0/180] [ 2613.231276] Key type lgssc registered [ 2613.772900] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2619.173502] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2622.596545] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2625.663472] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2628.960499] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2631.677122] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2634.808037] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2634.831386] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 2634.837061] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2635.949299] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2635.969932] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2636.011225] Lustre: lustre-MDT0000: new disk, initializing [ 2636.047343] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2636.059987] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2637.783381] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing set_default_debug -1 all [ 2643.359981] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2643.391936] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2643.515720] Lustre: lustre-OST0000: new disk, initializing [ 2643.521632] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 2643.526487] Lustre: Skipped 1 previous similar message [ 2643.564209] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2645.013601] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 2645.018317] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 2645.035482] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 2646.225946] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing set_default_debug -1 all [ 2651.913923] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2651.953596] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2652.033147] Lustre: lustre-OST0001: new disk, initializing [ 2652.036045] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 2652.075708] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 2654.672093] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing set_default_debug -1 all [ 2657.829471] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:1:ost [ 2657.834204] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:1:ost] [ 2657.848645] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x280000400 [ 2662.880812] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2662.881435] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2662.885777] Lustre: Skipped 1 previous similar message [ 2662.890127] Lustre: Skipped 1 previous similar message [ 2665.546805] LustreError: 114176:0:(obd_class.h:479:obd_check_dev()) Device 17 not setup [ 2665.654746] Lustre: server umount lustre-MDT0000 complete [ 2668.613960] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: (null) [ 2670.587704] LustreError: 112733:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1764793580 with bad export cookie 17904775607465096543 [ 2670.590117] LustreError: MGC192.168.202.124@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2670.592366] LustreError: 112733:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 2670.609118] LustreError: 114772:0:(obd_class.h:479:obd_check_dev()) Device 12 not setup [ 2670.612370] LustreError: 114772:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 2670.631662] Lustre: server umount lustre-OST0000 complete [ 2672.214928] LustreError: 114972:0:(obd_class.h:479:obd_check_dev()) Device 2 not setup [ 2672.218041] LustreError: 114972:0:(obd_class.h:479:obd_check_dev()) Skipped 1 previous similar message [ 2672.299148] Lustre: server umount lustre-OST0001 complete [ 2675.597572] Lustre: DEBUG MARKER: == conf-sanity test 88: check the default mount options can be overridden ========================================================== 15:26:25 (1764793585) [ 2679.252453] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing load_modules_local [ 2683.029174] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2683.489377] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2685.967982] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2686.489222] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2689.218733] Lustre: DEBUG MARKER: == conf-sanity test 89: check tunefs --param and --erase-param{s} options ========================================================== 15:26:39 (1764793599) [ 2695.200458] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2696.100354] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2697.216243] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2698.173536] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2698.644341] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2699.610533] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2706.323790] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing set_hostid [ 2710.226350] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing load_modules_local [ 2714.086523] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2716.543376] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2718.997531] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2721.499097] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2724.270547] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2724.298841] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2724.411411] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2724.427763] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2724.471782] Lustre: lustre-MDT0000: new disk, initializing [ 2724.510159] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2724.519220] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2726.199781] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing set_default_debug -1 all [ 2730.891556] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2730.915592] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2730.957751] Lustre: 120291:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2730.984567] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 2730.987072] Lustre: Skipped 1 previous similar message [ 2731.027422] Lustre: lustre-MDT0001: new disk, initializing [ 2731.080574] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2731.097788] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 2731.102151] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2732.678561] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing set_default_debug -1 all [ 2736.265628] Lustre: DEBUG MARKER: oleg224-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2737.740366] Lustre: DEBUG MARKER: oleg224-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2740.343304] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2740.374520] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2740.480105] Lustre: lustre-OST0000: new disk, initializing [ 2740.482885] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 2740.515927] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2742.778418] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing set_default_debug -1 all [ 2746.373909] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 2746.378682] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2746.416995] Lustre: DEBUG MARKER: oleg224-client.virtnet: executing wait_import_state_mount (FULL|IDLE) failover.node=192.0.2.254@tcp0,osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2746.422199] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2748.092889] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2748.187320] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2749.740988] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2749.836802] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2751.461302] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2751.461972] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2751.471771] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2751.474142] Lustre: Skipped 1 previous similar message [ 2752.623161] LustreError: 123232:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 2752.626416] LustreError: 123232:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 2752.702018] Lustre: server umount lustre-OST0000 complete [ 2756.576955] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2756.577069] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2756.580451] LustreError: Skipped 1 previous similar message [ 2756.585211] Lustre: Skipped 1 previous similar message [ 2756.585915] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2756.592709] Lustre: Skipped 1 previous similar message [ 2760.257695] LustreError: 123433:0:(obd_class.h:479:obd_check_dev()) Device 22 not setup [ 2760.260211] LustreError: 123433:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 2760.290559] Lustre: server umount lustre-MDT0000 complete [ 2761.726927] LustreError: 120282:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1764793671 with bad export cookie 17904775607465097747 [ 2761.732054] LustreError: MGC192.168.202.124@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2761.732079] LustreError: 120282:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2761.901621] Lustre: server umount lustre-MDT0001 complete [ 2765.150238] Lustre: DEBUG MARKER: == conf-sanity test 90a: check max_mod_rpcs_in_flight is enforced ========================================================== 15:27:55 (1764793675) [ 2767.706754] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2767.860775] LustreError: 124304:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2767.869123] LustreError: 124304:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 2767.885579] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2769.306244] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing set_default_debug -1 all [ 2772.624949] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2774.198227] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing set_default_debug -1 all [ 2776.195609] Lustre: DEBUG MARKER: oleg224-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2777.527224] Lustre: DEBUG MARKER: oleg224-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2780.069596] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2782.434278] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing set_default_debug -1 all [ 2784.531241] Lustre: DEBUG MARKER: oleg224-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2797.487317] Lustre: *** cfs_fail_loc=159, val=0*** [ 2813.016539] Lustre: lustre-MDT0000: Client 2f9a5175-2523-445d-bd7d-9b6d37e1bbfb (at 192.168.202.24@tcp) reconnecting [ 2813.606158] Lustre: *** cfs_fail_loc=159, val=0*** [ 2813.608041] Lustre: Skipped 5 previous similar messages [ 2829.903953] Lustre: lustre-MDT0000: Client 2f9a5175-2523-445d-bd7d-9b6d37e1bbfb (at 192.168.202.24@tcp) reconnecting [ 2849.763090] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2849.764359] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2849.767910] LustreError: Skipped 1 previous similar message [ 2849.768960] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2849.773393] Lustre: Skipped 2 previous similar messages [ 2851.131176] LustreError: 126814:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 2851.134963] LustreError: 126814:0:(obd_class.h:479:obd_check_dev()) Skipped 13 previous similar messages [ 2851.214228] Lustre: server umount lustre-OST0000 complete [ 2854.882090] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2854.882366] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2854.891729] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2854.894230] Lustre: Skipped 2 previous similar messages [ 2865.120438] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2865.123205] Lustre: Skipped 3 previous similar messages [ 2867.167136] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2867.209558] LustreError: 127017:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup [ 2867.212525] LustreError: 127017:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 2867.250874] Lustre: server umount lustre-MDT0000 complete [ 2868.685161] LustreError: 124284:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1764793778 with bad export cookie 17904775607465098615 [ 2868.687310] LustreError: MGC192.168.202.124@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2868.689256] LustreError: 124284:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 2868.851926] Lustre: server umount lustre-MDT0001 complete [ 2872.870209] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing unload_modules_local [ 2873.995517] Key type lgssc unregistered [ 2874.141464] LNet: 127751:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2874.144435] LNetError: 127751:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2874.153406] LNet: Removed LNI 192.168.202.124@tcp [ 2874.480051] Key type .llcrypt unregistered [ 2874.481905] Key type ._llcrypt unregistered [ 2884.846815] Key type ._llcrypt registered [ 2884.848288] Key type .llcrypt registered [ 2884.895192] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing load_modules_local [ 2885.378218] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2885.386958] alg: No test for adler32 (adler32-zlib) [ 2886.273202] Lustre: Lustre: Build Version: 2.16.59_23_g5b5e124 [ 2886.366104] LNet: Added LNI 192.168.202.124@tcp [8/256/0/180] [ 2887.951355] Key type lgssc registered [ 2888.413261] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2891.656786] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 2891.668243] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2892.873717] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2894.489921] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing set_default_debug -1 all [ 2897.964619] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2898.127696] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2899.644292] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing set_default_debug -1 all [ 2901.720551] Lustre: DEBUG MARKER: oleg224-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2903.079074] Lustre: DEBUG MARKER: oleg224-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2905.626085] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2905.757212] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2907.850879] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing set_default_debug -1 all [ 2910.000644] Lustre: DEBUG MARKER: oleg224-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2918.060753] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:42 to 0x280000401:65) [ 2925.944309] Lustre: *** cfs_fail_loc=159, val=0*** [ 2941.531737] Lustre: lustre-MDT0000: Client 90c3cb9d-e567-4697-891e-293c6885d633 (at 192.168.202.24@tcp) reconnecting [ 2941.648821] Lustre: 129120:0:(osd_internal.h:1360:osd_trans_exec_op()) lustre-MDT0001: opcode 2: before 507 < left 525, rollback = 2 [ 2941.654642] Lustre: 129120:0:(osd_handler.c:2081:osd_trans_dump_creds()) create: 3/12/6, destroy: 0/0/0 [ 2941.661895] Lustre: 129120:0:(osd_handler.c:2088:osd_trans_dump_creds()) attr_set: 3/3/0, xattr_set: 6/525/0 [ 2941.666052] Lustre: 129120:0:(osd_handler.c:2098:osd_trans_dump_creds()) write: 7/153/0, punch: 0/0/0, quota 1/3/0 [ 2941.669915] Lustre: 129120:0:(osd_handler.c:2105:osd_trans_dump_creds()) insert: 7/135/3, delete: 0/0/0 [ 2941.673798] Lustre: 129120:0:(osd_handler.c:2112:osd_trans_dump_creds()) ref_add: 1/1/0, ref_del: 0/0/0 [ 2942.544469] Lustre: *** cfs_fail_loc=159, val=0*** [ 2958.936317] Lustre: lustre-MDT0001: Client 90c3cb9d-e567-4697-891e-293c6885d633 (at 192.168.202.24@tcp) reconnecting [ 2959.543690] Lustre: *** cfs_fail_loc=159, val=0*** [ 2959.546294] Lustre: Skipped 3 previous similar messages [ 2975.827043] Lustre: lustre-MDT0001: Client 90c3cb9d-e567-4697-891e-293c6885d633 (at 192.168.202.24@tcp) reconnecting [ 2977.483827] Lustre: 131849:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 2978.747777] Lustre: *** cfs_fail_loc=159, val=0*** [ 2978.749666] Lustre: Skipped 1 previous similar message [ 2994.261254] Lustre: lustre-MDT0000: Client 94763a11-4b57-45da-b70e-2e0d17c6404b (at 192.168.202.24@tcp) reconnecting [ 2994.988198] Lustre: *** cfs_fail_loc=159, val=0*** [ 2994.990520] Lustre: Skipped 14 previous similar messages [ 3011.159416] Lustre: lustre-MDT0000: Client 94763a11-4b57-45da-b70e-2e0d17c6404b (at 192.168.202.24@tcp) reconnecting [ 3011.838399] Lustre: 132139:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 3015.648665] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3015.648890] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3015.652089] LustreError: Skipped 1 previous similar message [ 3015.658643] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3015.662486] Lustre: Skipped 1 previous similar message [ 3019.634120] LustreError: 132237:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 3019.704183] Lustre: server umount lustre-OST0000 complete [ 3021.280074] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3021.283657] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3021.289237] Lustre: Skipped 1 previous similar message [ 3021.291985] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3025.891856] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3025.895712] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3027.334969] LustreError: 132440:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup [ 3027.337985] LustreError: 132440:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 3029.410089] LustreError: 132440:0:(obd_class.h:479:obd_check_dev()) Device 10 not setup [ 3029.415617] LustreError: 132440:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 3029.437309] Lustre: server umount lustre-MDT0000 complete [ 3030.893384] LustreError: 131968:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1764793941 with bad export cookie 16705163323783603001 [ 3030.896060] LustreError: MGC192.168.202.124@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3030.896926] LustreError: 131968:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 3031.044314] Lustre: server umount lustre-MDT0001 complete [ 3034.653687] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing unload_modules_local [ 3035.624161] Key type lgssc unregistered [ 3035.741406] LNet: 133174:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3035.743823] LNetError: 133174:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3035.753393] LNet: Removed LNI 192.168.202.124@tcp [ 3036.045545] Key type .llcrypt unregistered [ 3036.047146] Key type ._llcrypt unregistered [ 3045.806692] Key type ._llcrypt registered [ 3045.809066] Key type .llcrypt registered [ 3045.864527] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing load_modules_local [ 3046.237090] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3046.284943] alg: No test for adler32 (adler32-zlib) [ 3047.149247] Lustre: Lustre: Build Version: 2.16.59_23_g5b5e124 [ 3047.256253] LNet: Added LNI 192.168.202.124@tcp [8/256/0/180] [ 3048.847243] Key type lgssc registered [ 3049.250808] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3051.991481] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3051.999385] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3053.157855] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3054.512692] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing set_default_debug -1 all [ 3057.552112] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3057.706124] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3059.093097] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing set_default_debug -1 all [ 3061.104083] Lustre: DEBUG MARKER: oleg224-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 3062.392579] Lustre: DEBUG MARKER: oleg224-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 3064.803817] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3064.927259] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3066.885537] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing set_default_debug -1 all [ 3066.985098] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:97) [ 3068.847758] Lustre: DEBUG MARKER: oleg224-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3079.271333] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:129) [ 3086.659906] Lustre: 136818:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 3087.352834] Lustre: 136866:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 3087.847774] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 in log params [ 3089.037960] Lustre: 137013:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify lustre/mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 (mode = 0) failed: rc = -17 [ 3089.359933] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=7 in log params [ 3089.362671] Lustre: Skipped 1 previous similar message [ 3089.665267] Lustre: 137111:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 3092.959862] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3092.962604] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3092.967783] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3095.009405] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3095.010627] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3095.011048] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3095.012132] Lustre: Skipped 1 previous similar message [ 3096.749083] LustreError: 137208:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 3096.813496] Lustre: server umount lustre-OST0000 complete [ 3098.591846] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3098.593938] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3098.598171] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3104.188263] LustreError: 137418:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup [ 3104.190610] LustreError: 137418:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 3104.219082] Lustre: server umount lustre-MDT0000 complete [ 3105.248069] LustreError: 134547:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3105.255692] LustreError: 134547:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 3105.528356] LustreError: 136916:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1764794015 with bad export cookie 16254817123119406518 [ 3105.530782] LustreError: MGC192.168.202.124@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3105.532912] LustreError: 136916:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 3105.609345] LustreError: 137620:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [ 3105.612414] LustreError: 137620:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 3105.691044] Lustre: server umount lustre-MDT0001 complete [ 3109.320599] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing unload_modules_local [ 3110.427733] Key type lgssc unregistered [ 3110.553376] LNet: 138150:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3110.556406] LNetError: 138150:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3110.566353] LNet: Removed LNI 192.168.202.124@tcp [ 3110.889770] Key type .llcrypt unregistered [ 3110.891041] Key type ._llcrypt unregistered [ 3120.216820] Key type ._llcrypt registered [ 3120.218155] Key type .llcrypt registered [ 3120.265547] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing load_modules_local [ 3120.748183] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3120.759561] alg: No test for adler32 (adler32-zlib) [ 3121.608375] Lustre: Lustre: Build Version: 2.16.59_23_g5b5e124 [ 3121.687799] LNet: Added LNI 192.168.202.124@tcp [8/256/0/180] [ 3123.271170] Key type lgssc registered [ 3123.619629] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3126.084285] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3126.091315] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3127.240092] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3128.558275] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing set_default_debug -1 all [ 3131.561940] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3131.712460] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3133.039861] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing set_default_debug -1 all [ 3135.030359] Lustre: DEBUG MARKER: oleg224-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 3136.340538] Lustre: DEBUG MARKER: oleg224-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 3138.710226] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3138.825893] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3140.772565] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing set_default_debug -1 all [ 3140.900793] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:129) [ 3142.694627] Lustre: DEBUG MARKER: oleg224-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3152.166382] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:161) [ 3155.704435] Lustre: *** cfs_fail_loc=159, val=0*** [ 3171.926576] Lustre: lustre-MDT0000: Client e50a59ce-597c-4501-badb-03dd37e44903 (at 192.168.202.24@tcp) reconnecting [ 3187.167765] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3187.170692] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3187.175217] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3188.193907] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3188.195083] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3188.195171] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3192.878122] LustreError: 141921:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 3192.945709] Lustre: server umount lustre-OST0000 complete [ 3198.434440] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3198.434939] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3198.446235] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3198.453102] Lustre: Skipped 2 previous similar messages [ 3200.714188] LustreError: 142123:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup [ 3200.719792] LustreError: 142123:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 3200.768946] Lustre: server umount lustre-MDT0000 complete [ 3202.801341] LustreError: 139502:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1764794113 with bad export cookie 11087383709217027948 [ 3202.802574] LustreError: MGC192.168.202.124@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3202.806191] LustreError: 139502:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 3202.889849] LustreError: 142324:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [ 3202.891967] LustreError: 142324:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 3203.010040] Lustre: server umount lustre-MDT0001 complete [ 3208.749789] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing unload_modules_local [ 3210.649192] Key type lgssc unregistered [ 3210.885178] LNet: 142854:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3210.892106] LNetError: 142854:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3211.943941] LNet: Removed LNI 192.168.202.124@tcp [ 3212.517835] Key type .llcrypt unregistered [ 3212.522327] Key type ._llcrypt unregistered [ 3227.898075] Key type ._llcrypt registered [ 3227.899891] Key type .llcrypt registered [ 3227.953789] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing load_modules_local [ 3228.548678] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3228.559258] alg: No test for adler32 (adler32-zlib) [ 3229.544048] Lustre: Lustre: Build Version: 2.16.59_23_g5b5e124 [ 3229.695727] LNet: Added LNI 192.168.202.124@tcp [8/256/0/180] [ 3231.319354] Key type lgssc registered [ 3232.058732] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3236.414514] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3236.423732] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3237.634968] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3239.193865] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing set_default_debug -1 all [ 3242.627373] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3242.799798] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3244.362748] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing set_default_debug -1 all [ 3246.609886] Lustre: DEBUG MARKER: oleg224-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 3248.150891] Lustre: DEBUG MARKER: oleg224-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 3251.091204] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3251.250419] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3253.290396] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:161) [ 3253.716650] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing set_default_debug -1 all [ 3255.887556] Lustre: DEBUG MARKER: oleg224-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3262.520878] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:170 to 0x280000401:193) [ 3279.977554] Lustre: lustre-OST0000: evicting 23d97898-ef1f-403d-9c4c-358ad53dbbe4 (at 192.168.202.24@tcp) by administrative request [ 3279.982270] Lustre: Skipped 1 previous similar message [ 3308.517109] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3308.518455] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3308.527660] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3308.529828] Lustre: Skipped 1 previous similar message [ 3310.511926] LustreError: 147032:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 3310.584611] Lustre: server umount lustre-OST0000 complete [ 3313.635159] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3313.635877] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3313.640449] Lustre: Skipped 2 previous similar messages [ 3318.341158] LustreError: 147234:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup [ 3318.344174] LustreError: 147234:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 3318.377136] Lustre: server umount lustre-MDT0000 complete [ 3318.752679] LustreError: 145392:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3318.759300] LustreError: 145392:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 3319.786289] LustreError: 144209:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1764794230 with bad export cookie 2867658687194255739 [ 3319.789380] LustreError: MGC192.168.202.124@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3319.791815] LustreError: 144209:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 3319.858830] LustreError: 147435:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [ 3319.860823] LustreError: 147435:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 3319.946459] Lustre: server umount lustre-MDT0001 complete [ 3323.841868] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing unload_modules_local [ 3324.967790] Key type lgssc unregistered [ 3325.104479] LNet: 147966:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3325.107308] LNetError: 147966:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3325.117462] LNet: Removed LNI 192.168.202.124@tcp [ 3325.465143] Key type .llcrypt unregistered [ 3325.467056] Key type ._llcrypt unregistered [ 3358.786593] Key type ._llcrypt registered [ 3358.788011] Key type .llcrypt registered [ 3358.884431] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing load_modules_local [ 3359.483515] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3359.496416] alg: No test for adler32 (adler32-zlib) [ 3360.518904] Lustre: Lustre: Build Version: 2.16.59_23_g5b5e124 [ 3360.684379] LNet: Added LNI 192.168.202.124@tcp [8/256/0/180] [ 3362.319184] Key type lgssc registered [ 3363.392835] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3373.219621] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3373.261672] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3374.769813] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3378.743550] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing set_default_debug -1 all [ 3380.199605] LustreError: 150413:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3380.219090] LustreError: 150413:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 3384.292672] LustreError: 150413:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3387.557149] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3387.834832] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3390.987366] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing set_default_debug -1 all [ 3395.267391] Lustre: DEBUG MARKER: oleg224-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 3398.607666] Lustre: DEBUG MARKER: oleg224-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 3403.689205] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3403.926304] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3407.657365] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing set_default_debug -1 all [ 3410.963384] Lustre: DEBUG MARKER: oleg224-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3412.152930] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:193) [ 3419.304321] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:195 to 0x280000401:225) [ 3427.808294] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3427.813068] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3427.819717] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3429.857540] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3429.858374] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3429.858786] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3429.873401] Lustre: Skipped 1 previous similar message [ 3431.858133] LustreError: 152689:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 3431.975825] Lustre: server umount lustre-OST0000 complete [ 3434.977662] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3434.981197] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3434.984916] Lustre: Skipped 1 previous similar message [ 3440.096231] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3440.102665] Lustre: Skipped 2 previous similar messages [ 3445.216691] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3445.220447] Lustre: Skipped 1 previous similar message [ 3448.287147] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 3448.376260] LustreError: 152890:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup [ 3448.378735] LustreError: 152890:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 3448.442075] Lustre: server umount lustre-MDT0000 complete [ 3450.336612] LustreError: 151128:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3450.838714] LustreError: 150394:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1764794361 with bad export cookie 15204405999891800457 [ 3450.847753] LustreError: MGC192.168.202.124@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3450.852421] LustreError: 150394:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 3450.953931] LustreError: 153094:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [ 3450.956325] LustreError: 153094:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 3451.118985] Lustre: server umount lustre-MDT0001 complete [ 3456.345130] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing unload_modules_local [ 3457.802557] Key type lgssc unregistered [ 3457.951075] LNet: 153627:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3457.956407] LNetError: 153627:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3457.972451] LNet: Removed LNI 192.168.202.124@tcp [ 3458.408307] Key type .llcrypt unregistered [ 3458.410594] Key type ._llcrypt unregistered [ 3471.456167] Key type ._llcrypt registered [ 3471.457866] Key type .llcrypt registered [ 3471.530595] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing load_modules_local [ 3472.055584] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3472.067944] alg: No test for adler32 (adler32-zlib) [ 3472.992534] Lustre: Lustre: Build Version: 2.16.59_23_g5b5e124 [ 3473.107744] LNet: Added LNI 192.168.202.124@tcp [8/256/0/180] [ 3474.728798] Key type lgssc registered [ 3475.307543] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3478.839665] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3482.475295] Lustre: DEBUG MARKER: == conf-sanity test 100: check lshowmount lists MGS, MDT, OST and 0@lo ========================================================== 15:39:52 (1764794392) [ 3490.030428] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing set_hostid [ 3494.215799] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing load_modules_local [ 3498.498806] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3501.375033] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3504.122166] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3506.997289] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3510.101898] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3510.129735] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3510.142056] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3511.251730] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3511.273442] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3511.316597] Lustre: lustre-MDT0000: new disk, initializing [ 3511.363813] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3511.373036] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3513.302041] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing set_default_debug -1 all [ 3518.218420] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3518.245887] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3518.289493] Lustre: 157821:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 3518.324929] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 3518.327323] Lustre: Skipped 1 previous similar message [ 3518.367039] Lustre: lustre-MDT0001: new disk, initializing [ 3518.409708] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3518.428859] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 3518.433374] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3520.040509] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing set_default_debug -1 all [ 3523.702608] Lustre: DEBUG MARKER: oleg224-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 3525.288472] Lustre: DEBUG MARKER: oleg224-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 3528.409220] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3528.440267] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3528.548859] Lustre: lustre-OST0000: new disk, initializing [ 3528.551686] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3528.588871] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3530.816307] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing set_default_debug -1 all [ 3534.348129] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 3534.353610] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 3534.407764] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3534.491486] Lustre: DEBUG MARKER: oleg224-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3539.425323] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3539.428695] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3539.431333] LustreError: Skipped 1 previous similar message [ 3539.433085] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3539.438770] Lustre: Skipped 1 previous similar message [ 3540.786113] LustreError: 160406:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 3540.877349] Lustre: server umount lustre-OST0000 complete [ 3544.033303] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3544.039718] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3544.046888] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3544.049894] Lustre: Skipped 1 previous similar message [ 3544.923118] LustreError: 160608:0:(obd_class.h:479:obd_check_dev()) Device 22 not setup [ 3544.931315] LustreError: 160608:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 3545.015690] Lustre: server umount lustre-MDT0000 complete [ 3546.638585] LustreError: 157812:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1764794456 with bad export cookie 17296843499706361737 [ 3546.641585] LustreError: MGC192.168.202.124@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3546.643633] LustreError: 157812:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 3546.720329] LustreError: 160808:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 3546.722972] LustreError: 160808:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 3546.827058] Lustre: server umount lustre-MDT0001 complete [ 3550.996306] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing unload_modules_local [ 3552.315901] Key type lgssc unregistered [ 3552.465557] LNet: 161339:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3552.469305] LNetError: 161339:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3552.477440] LNet: Removed LNI 192.168.202.124@tcp [ 3552.865385] Key type .llcrypt unregistered [ 3552.867087] Key type ._llcrypt unregistered [ 3563.721931] Key type ._llcrypt registered [ 3563.723564] Key type .llcrypt registered [ 3563.763069] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing load_modules_local [ 3564.171108] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3564.223811] alg: No test for adler32 (adler32-zlib) [ 3565.103203] Lustre: Lustre: Build Version: 2.16.59_23_g5b5e124 [ 3565.199647] LNet: Added LNI 192.168.202.124@tcp [8/256/0/180] [ 3566.791211] Key type lgssc registered [ 3567.200477] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3570.244592] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3570.254867] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3571.464653] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3573.044857] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing set_default_debug -1 all [ 3576.484846] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3576.630189] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3578.054776] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing set_default_debug -1 all [ 3580.129558] Lustre: DEBUG MARKER: oleg224-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 3581.546186] Lustre: DEBUG MARKER: oleg224-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 3584.188649] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3584.318449] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3586.478125] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing set_default_debug -1 all [ 3588.504126] Lustre: DEBUG MARKER: oleg224-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3596.646600] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:3 to 0x280000401:33) [ 3601.518633] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 3601.527054] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3601.533048] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 3601.536609] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3601.541585] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 3601.901075] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3602.280140] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 3602.282875] Lustre: Skipped 1 previous similar message [ 3602.289612] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3602.295036] Lustre: Skipped 1 previous similar message [ 3602.297534] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 3602.299918] Lustre: Skipped 1 previous similar message [ 3602.301874] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3602.306054] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 3602.307960] Lustre: Skipped 1 previous similar message [ 3602.635608] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3603.011741] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3603.382652] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 3603.386193] Lustre: Skipped 2 previous similar messages [ 3603.395226] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3603.400691] Lustre: Skipped 2 previous similar messages [ 3603.403156] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 3603.405580] Lustre: Skipped 2 previous similar messages [ 3603.407685] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3603.412870] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 3603.415516] Lustre: Skipped 2 previous similar messages [ 3603.759552] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3604.076700] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3604.394382] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3604.740807] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3605.088202] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3605.403233] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 3605.406820] Lustre: Skipped 5 previous similar messages [ 3605.413889] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3605.419254] Lustre: Skipped 5 previous similar messages [ 3605.421027] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 3605.423163] Lustre: Skipped 5 previous similar messages [ 3605.424417] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3605.428674] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 3605.431865] Lustre: Skipped 5 previous similar messages [ 3605.733380] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3606.080013] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3606.370720] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3606.676451] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3606.990794] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3607.317819] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3607.642274] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3607.933395] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3608.240053] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3608.553852] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3608.864724] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3609.164467] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3609.464330] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 3609.466247] Lustre: Skipped 12 previous similar messages [ 3609.471115] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3609.474438] Lustre: Skipped 12 previous similar messages [ 3609.476103] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 3609.477841] Lustre: Skipped 12 previous similar messages [ 3609.479197] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3609.483272] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 3609.485436] Lustre: Skipped 12 previous similar messages [ 3609.793449] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3610.114907] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3610.459028] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3610.785349] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3611.126261] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3611.464983] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3611.794636] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3612.113070] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3612.441066] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3612.753831] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3613.067440] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3613.371351] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3613.691131] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3614.011763] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3614.331929] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3614.652763] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3614.968682] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3615.270337] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3615.567986] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3615.863857] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3616.174547] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3616.485267] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3616.806391] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3617.112436] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3617.431533] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3617.741539] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 3617.743946] Lustre: Skipped 25 previous similar messages [ 3617.749342] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3617.753228] Lustre: Skipped 25 previous similar messages [ 3617.754747] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 3617.757335] Lustre: Skipped 25 previous similar messages [ 3617.758820] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3617.762341] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 3617.764234] Lustre: Skipped 25 previous similar messages [ 3618.074457] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3618.416840] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3618.766507] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3619.103638] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3619.452733] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3619.800291] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3620.137435] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3620.463516] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3620.801953] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3621.153067] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3621.465131] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3621.805336] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3622.130132] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3622.485600] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3622.840215] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3623.188983] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3623.573151] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3623.920648] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3624.263535] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3624.605444] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3624.922866] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3625.246148] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3625.575127] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3625.884722] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3626.198993] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3626.509694] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3626.841921] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3627.142585] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3627.450758] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3627.762016] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3628.073066] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3628.367326] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3628.677696] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3628.984337] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3629.306284] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3629.632880] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3629.953354] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3630.272075] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3630.618334] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3630.938712] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3631.259324] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3631.594303] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3631.928504] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3632.231913] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3632.520589] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3632.821871] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3633.126947] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3633.435388] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3633.736686] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3634.054829] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 3634.057499] Lustre: Skipped 49 previous similar messages [ 3634.063974] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3634.069153] Lustre: Skipped 49 previous similar messages [ 3634.071656] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 3634.073637] Lustre: Skipped 49 previous similar messages [ 3634.075242] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3634.079323] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 3634.081424] Lustre: Skipped 49 previous similar messages [ 3634.402869] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3634.742273] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3635.093595] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3635.450331] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3635.821552] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3636.171953] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3636.514112] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3636.855803] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3637.199996] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3637.572726] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3637.938891] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3638.292901] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3638.612887] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3638.931158] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3639.266823] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3639.595087] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3639.922520] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3640.264140] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3640.582330] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3640.933952] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3641.256252] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3641.570987] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3641.883278] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3642.226986] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3642.570388] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3642.926728] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3643.262804] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3643.627604] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3643.980763] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3644.336698] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3644.669964] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3645.000953] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3645.324140] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3645.646491] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3645.968854] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3646.294836] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3646.615378] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3646.945805] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3647.277593] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3647.663602] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3648.017090] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3648.368906] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3648.724463] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3649.083340] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3649.440420] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3649.798506] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3650.142259] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3650.482541] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3650.831388] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3651.197221] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3651.521276] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3651.856167] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3652.187419] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3652.483681] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3652.793880] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3653.130519] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3653.452470] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3653.777512] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3654.101304] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3654.445743] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3654.762271] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3655.083085] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3655.382408] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3655.712284] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3656.036595] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3656.359353] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3656.678598] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3657.006390] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3657.348255] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3657.668925] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3657.973013] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3658.304346] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3658.625249] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3658.949860] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3659.275230] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3659.624420] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3659.944566] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3660.272364] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3660.604453] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3660.944690] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3661.296261] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3661.612635] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3661.927308] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3662.272641] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3662.607190] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3662.935909] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3663.278027] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3663.644331] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3664.002789] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3664.370148] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3664.719610] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3665.062023] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3665.400890] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3665.719371] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3666.055670] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3666.371728] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 3666.374637] Lustre: Skipped 95 previous similar messages [ 3666.381303] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3666.385979] Lustre: Skipped 95 previous similar messages [ 3666.387938] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 3666.391099] Lustre: Skipped 95 previous similar messages [ 3666.393188] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3666.398190] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 3666.400845] Lustre: Skipped 95 previous similar messages [ 3666.715458] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3667.035703] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3667.362819] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3667.688739] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3668.016985] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3668.341743] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3668.656503] LustreError: 162731:0:(osp_precreate.c:651:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 3668.660982] LustreError: 162731:0:(osp_precreate.c:1355:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 3668.661255] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3669.006252] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3669.343942] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3669.687247] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3670.031996] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3670.383227] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3670.723459] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3671.072467] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3671.400314] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3671.734195] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3672.047529] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3672.385160] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3672.715797] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3673.031096] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3673.365713] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3673.688316] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3674.009958] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3674.355438] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3674.690916] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3675.024850] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3675.361155] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3675.687943] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3676.019533] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3676.349682] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3676.659708] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3676.983109] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3677.297863] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3677.633807] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3677.962294] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3678.294668] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3678.607573] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3678.919798] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3679.223109] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3679.561280] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3679.861900] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3680.167045] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3680.499211] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3680.841386] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3681.151612] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3681.502292] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3681.889117] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3682.218775] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3682.549549] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3682.862874] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3683.198478] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3683.521808] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3683.831715] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3684.176066] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3684.511232] LustreError: 162731:0:(osp_precreate.c:651:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 3684.515540] LustreError: 162731:0:(osp_precreate.c:1355:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 3684.516677] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3684.834985] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3685.166622] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3685.465504] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3685.770399] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3686.061738] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3686.367131] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3686.677715] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3687.012492] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3687.337534] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3687.672899] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3687.979598] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3688.291697] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3688.590509] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3688.909271] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3689.204816] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3689.550873] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3689.857067] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3690.169675] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3690.504275] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3690.820930] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3691.124037] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3691.433620] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3691.739137] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3692.063097] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3692.366058] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3692.652957] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3692.957565] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3693.290105] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3693.607910] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3693.928215] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3694.244306] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3694.580815] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3694.887228] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3695.202662] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3695.525802] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3695.866932] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3696.208591] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3696.543891] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3696.925697] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3697.280598] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3697.605375] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3697.930845] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3698.266606] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3698.614296] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3698.943498] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3699.264327] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3699.660428] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3700.007848] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3700.372672] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3700.731096] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3701.080041] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3701.457258] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3701.820262] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3702.149159] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3702.478699] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3702.817640] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3703.191593] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3703.540258] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3703.908333] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3704.280423] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3704.648744] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3705.003236] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3705.372632] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3705.815027] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3706.248559] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3706.719781] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3707.189258] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3707.666981] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3708.210941] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3708.771431] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3709.330679] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3709.856883] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3710.398912] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3711.139890] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3711.957128] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3712.949360] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3713.954147] LustreError: 162731:0:(osp_precreate.c:651:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 3713.961617] LustreError: 162731:0:(osp_precreate.c:1355:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 3713.968300] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3714.856063] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3715.571310] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3716.266225] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3717.128270] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3717.986906] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3718.739746] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3719.455324] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3720.404726] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3721.341145] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3722.256065] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3723.060651] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3724.064273] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3724.868334] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3725.906261] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3726.847045] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3727.922287] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3728.784192] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3729.699924] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3730.700575] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 3730.703123] Lustre: Skipped 150 previous similar messages [ 3730.737255] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3730.748744] Lustre: Skipped 150 previous similar messages [ 3730.756383] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 3730.764695] Lustre: Skipped 150 previous similar messages [ 3730.774047] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3730.787353] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 3730.798238] Lustre: Skipped 150 previous similar messages [ 3731.710228] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3732.588682] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3733.601372] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3734.477956] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3735.528278] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3736.312125] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3737.120548] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3738.243211] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3739.169252] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3740.172318] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3741.108051] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3742.048672] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3743.012826] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3743.857541] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3744.755695] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3745.557684] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3746.425535] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3747.660355] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3748.512654] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3749.472413] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3750.555306] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3751.888152] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3752.891075] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3753.802380] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3754.712381] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3755.458969] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3756.340094] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3757.235772] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3758.128149] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3758.845632] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3759.752035] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3760.464809] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3761.312824] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3762.235867] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3763.188371] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3764.020598] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3765.015219] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3765.791436] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3766.485553] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3767.208419] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3767.997474] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3768.905833] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3769.581972] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3770.167461] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3770.764916] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3771.362206] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3771.981277] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3772.529977] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3773.126654] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3773.631844] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3774.147442] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3774.656308] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3775.178576] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3775.763302] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3776.347775] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3776.894307] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3777.395931] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3778.014662] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3778.494778] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3778.974353] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3779.506435] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3780.096655] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3780.724331] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3781.267769] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3781.765945] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3782.334414] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3782.818201] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3783.335376] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3783.862715] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3784.393543] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3784.961691] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3785.441197] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3785.985332] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3786.587864] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3787.190145] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3787.717118] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3788.254902] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3788.717434] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3789.154982] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3789.597957] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3790.059643] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3790.533846] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3790.970603] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3791.368678] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3791.787535] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3792.212495] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3792.631456] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3793.088432] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3793.514633] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3793.978681] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3794.421843] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3794.881416] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3795.404651] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3795.943563] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3796.457209] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3796.985091] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3797.534795] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3797.971595] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3798.370214] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3798.847274] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3799.257702] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3799.692227] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3800.251316] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3800.678781] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3801.083101] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3801.513766] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3801.934759] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3802.327145] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3802.766439] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3803.179716] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3803.636620] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3804.127866] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3804.624947] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3805.148246] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3805.675469] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3806.205083] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3806.767408] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3807.270786] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3807.774122] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3808.358194] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3808.918387] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3809.457378] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3810.051844] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3810.525617] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3811.010297] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3811.525337] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3812.030319] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3812.507356] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3813.022946] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3813.517043] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3814.028919] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3814.522863] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3814.993883] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3815.524775] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3815.962810] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3816.441696] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3817.110463] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3817.729833] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3818.309770] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3818.908873] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3819.568667] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3820.144830] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3820.769453] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3821.340500] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3821.948189] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3822.504694] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3823.125102] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3823.647435] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3824.228337] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3824.871369] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3825.552364] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3826.160299] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3826.812253] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3827.315758] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3827.830883] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3828.348475] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3828.920598] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3829.479075] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3830.094601] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3830.737119] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3831.306628] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3831.878530] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3832.373423] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3832.835684] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3833.277208] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3833.756787] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3834.339812] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3834.862975] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3835.353661] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3835.886239] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3836.415169] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3836.939590] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3837.566311] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3838.036508] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3838.506065] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3839.028314] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3839.501733] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3839.919867] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3840.323402] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3840.769139] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3841.257054] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3841.749870] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3842.252208] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3842.684055] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3843.202250] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3843.740471] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3844.193310] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3844.664363] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3845.145291] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3845.546873] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3846.051618] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3846.556302] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3847.056914] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3847.505889] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3847.936328] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3848.381928] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3848.840690] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3849.282552] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3849.770409] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3850.248827] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3850.681265] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3851.124664] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3851.597033] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3852.162908] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3852.733390] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3853.435343] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3854.057914] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3854.613241] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3855.230948] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3855.838612] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3856.398247] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3857.059138] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3857.705533] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3858.549204] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3859.558442] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 3859.563283] Lustre: Skipped 214 previous similar messages [ 3859.582840] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3859.596886] Lustre: Skipped 214 previous similar messages [ 3859.600290] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 3859.602627] Lustre: Skipped 214 previous similar messages [ 3859.611185] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3859.618877] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 3859.621023] Lustre: Skipped 214 previous similar messages [ 3860.527846] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3861.646702] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3862.687408] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3863.912122] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3865.165896] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3866.467967] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3867.636637] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3868.789136] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3869.925084] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3870.885783] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3872.080014] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3873.091821] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3874.329460] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3875.662439] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3876.662039] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3877.848442] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3879.060727] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3880.228782] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3881.619805] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3882.844722] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3883.863757] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3885.064663] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3886.241501] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3887.389065] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3888.578156] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3889.781655] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3890.855706] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3892.039296] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3893.295966] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3894.580222] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3895.863893] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3897.142641] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3898.915351] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3900.377079] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3901.623267] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3902.698953] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3903.690335] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3904.902700] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3906.214644] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3907.380575] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3908.433456] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3909.524807] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3910.550314] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3911.641490] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3912.604351] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3913.591959] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3914.273217] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3915.168174] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3915.944787] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3916.959661] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3917.846986] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3918.584843] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3919.366242] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3920.265857] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3921.131345] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3921.903954] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3922.827130] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3923.705622] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3924.456777] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3925.246837] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3926.070530] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3926.802863] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3927.591369] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3928.254942] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3928.959280] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3929.642953] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3930.379751] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3931.050200] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3931.793915] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3932.443629] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3933.230861] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3934.144156] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3934.871775] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3935.762161] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3936.735137] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3937.494856] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3938.392895] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3939.391967] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3940.260601] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3941.107405] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3941.900810] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3942.598353] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3943.397069] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3944.336938] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3945.399913] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3946.285769] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3947.066187] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3947.980996] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3948.993433] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3949.849805] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3950.768465] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3951.644848] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3952.309143] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3953.224084] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3954.300476] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3955.223766] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3956.096301] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3956.901280] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3957.747348] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3958.721364] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3959.598883] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3960.453725] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3961.306843] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3962.095603] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3962.794819] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3963.530668] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3964.190572] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3965.021407] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3965.773382] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3966.434445] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3967.147964] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3967.893987] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3968.467494] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3969.180750] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3969.846741] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3970.545495] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3971.157707] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3971.760465] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3972.429108] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3973.042736] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3973.647864] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3974.284615] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3974.843878] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3975.461985] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3976.162296] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3976.805562] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3977.523220] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3978.188506] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3978.871657] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3979.601648] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3980.373164] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3981.023812] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3981.728091] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3982.385108] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3983.064759] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3983.766081] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3984.474329] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3985.170446] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3986.040811] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3986.692722] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3987.413712] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3988.183667] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3988.943893] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3989.607951] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3990.407322] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3991.109401] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3991.859035] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3992.608730] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3993.368045] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3993.912878] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3994.603662] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3995.399812] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3996.041081] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3996.760490] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3997.492919] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3998.413734] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3999.288598] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4000.108243] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4000.943341] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4001.707969] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4002.429687] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4003.285958] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4004.223813] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4005.179576] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4006.426834] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4007.376799] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4008.271358] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4009.265362] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4010.173469] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4011.240730] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4012.089933] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4013.185509] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4014.031437] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4014.919128] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4015.793356] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4016.883762] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4017.761993] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4018.679603] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4019.672392] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4020.679547] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4021.454993] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4022.286444] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4022.965915] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4023.574762] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4024.199506] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4024.966949] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4025.719186] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4026.466612] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4027.160669] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4027.920723] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4028.505896] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4029.193242] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4029.864055] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4030.459562] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4031.101459] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4031.621214] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4032.156154] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4032.764308] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4033.377307] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4034.093712] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4034.736614] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4035.300357] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4035.962820] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4036.665324] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4037.351308] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4037.959390] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4038.502579] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4039.055602] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4039.679886] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4040.402029] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4041.110121] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4041.792290] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4042.556360] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4043.341038] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4043.996260] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4044.578119] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4045.088404] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4045.663535] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4046.144087] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4046.691231] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4047.248371] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4047.732840] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4048.226292] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4048.711208] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4049.175934] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4049.627978] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4050.076868] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4050.574982] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4051.101569] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4051.714319] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4052.268343] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4052.751323] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4053.289249] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4053.744484] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4054.144081] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4054.588472] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4055.048052] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4055.518360] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4055.907922] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4056.305684] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4056.708863] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4057.085233] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4057.460800] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4057.862869] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4058.262596] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4058.646286] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4059.064854] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4059.475376] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4059.862543] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4060.263173] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4060.633967] LustreError: 162731:0:(osp_precreate.c:651:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 4060.638381] LustreError: 162731:0:(osp_precreate.c:1355:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 4060.641226] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4061.050082] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4061.458246] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4061.889540] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4062.330281] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4062.822421] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4063.306488] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4063.815174] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4064.306613] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4064.866238] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4065.447754] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4065.980307] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4066.486139] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4067.014384] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4067.561706] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4068.076727] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4068.667141] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4069.223497] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4069.708064] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4070.255723] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4070.792299] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4071.301580] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4071.783614] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4072.307929] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4072.786644] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4073.317760] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4073.829300] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4074.362806] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4074.975715] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4075.492508] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4075.936873] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4076.421707] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4076.941573] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4077.461088] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4077.977367] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4078.402885] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4078.787302] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4079.187990] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4079.625622] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4080.048402] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4080.481957] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4080.901352] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4081.332536] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4081.785445] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4082.217878] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4082.705345] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4083.155193] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4083.631885] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4084.100986] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4084.566555] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4085.025916] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4085.497390] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4085.941173] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4086.495093] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4087.012899] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4087.569242] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4088.155819] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4088.681817] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4089.278619] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4089.873657] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4090.457930] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4091.094218] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4091.660959] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4092.173255] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4092.691175] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4093.234162] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4093.794141] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4094.391424] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4094.972287] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4095.524410] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4096.177433] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4096.913208] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4097.667115] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4098.304222] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4098.956063] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4099.592077] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4100.224716] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4100.822411] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4101.379606] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4101.897706] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4102.422177] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4103.021946] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4103.545974] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4104.091668] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4104.664498] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4105.223591] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4105.720476] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4106.359695] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4107.032749] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4107.606856] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4108.126076] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4108.609656] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4109.141823] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4109.619914] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4110.064387] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4110.630459] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4111.243870] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4111.764723] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4112.279363] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4112.833114] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4113.352529] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4113.855929] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4114.286638] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4114.807066] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4115.287261] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4115.751070] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4115.754275] Lustre: Skipped 355 previous similar messages [ 4115.761627] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4115.766976] Lustre: Skipped 355 previous similar messages [ 4115.769889] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4115.773153] Lustre: Skipped 355 previous similar messages [ 4115.776689] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4115.781974] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4115.784781] Lustre: Skipped 355 previous similar messages [ 4116.265469] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4116.775814] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4117.335338] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4117.837378] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4118.310150] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4118.843669] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4119.370735] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4119.958741] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4120.589414] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4121.358098] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4122.030365] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4122.620241] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4123.195472] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4123.799894] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4124.632900] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4125.443629] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4126.500176] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4127.438682] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4128.595394] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4129.618695] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4130.916676] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4132.191738] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4133.174649] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4134.410593] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4135.753561] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4136.930761] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4138.177496] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4139.476675] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4140.672523] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4141.586110] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4142.892114] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4143.878188] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4144.954119] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4145.913640] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4146.840972] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4147.800181] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4148.952947] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4149.999478] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4150.989840] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4152.057025] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4153.019261] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4154.084593] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4155.156630] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4156.018469] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4157.033827] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4158.084140] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4159.304966] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4160.310288] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4161.335980] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4162.489585] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4163.392500] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4164.362200] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4165.360353] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4166.387205] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4167.697323] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4168.847268] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4169.889687] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4171.171398] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4172.097775] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4173.442822] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4174.599931] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4175.775785] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4176.718792] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4177.966954] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4179.073473] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4180.187621] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4180.965200] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4181.865944] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4182.644140] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4183.496580] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4184.285413] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4185.681813] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4186.595897] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4187.642631] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4188.592066] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4189.553105] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4190.303847] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4191.183851] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4191.882992] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4192.673620] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4193.467877] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4712.488756] LustreError: 164307:0:(ofd_dev.c:1780:ofd_create_hdl()) lustre-OST0000: unable to precreate: rc = -28 [ 4712.496155] LustreError: 162731:0:(osp_precreate.c:651:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -28 [ 5433.826083] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5433.826835] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5433.836354] Lustre: Skipped 82 previous similar messages [ 5433.841938] Lustre: Skipped 1 previous similar message [ 5438.537900] LustreError: 216694:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 5438.828687] Lustre: server umount lustre-OST0000 complete [ 5444.068220] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5444.079767] Lustre: Skipped 1 previous similar message [ 5448.309863] LustreError: 216898:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup [ 5448.321675] LustreError: 216898:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 5448.530568] Lustre: server umount lustre-MDT0000 complete [ 5449.189486] LustreError: 162711:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5449.215101] LustreError: 162711:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 5451.833515] LustreError: 162692:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1764796362 with bad export cookie 7439771443837288518 [ 5451.836384] LustreError: MGC192.168.202.124@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5451.843184] LustreError: 162692:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 5451.981405] LustreError: 217099:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [ 5451.985668] LustreError: 217099:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 5452.227863] Lustre: server umount lustre-MDT0001 complete [ 5460.980190] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing unload_modules_local [ 5462.992921] Key type lgssc unregistered [ 5463.202785] LNet: 217630:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5463.214136] LNetError: 217630:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5463.226441] LNet: Removed LNI 192.168.202.124@tcp [ 5463.841324] Key type .llcrypt unregistered [ 5463.843248] Key type ._llcrypt unregistered [ 5486.348852] Key type ._llcrypt registered [ 5486.350559] Key type .llcrypt registered [ 5486.455678] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing load_modules_local [ 5487.334230] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5487.406972] alg: No test for adler32 (adler32-zlib) [ 5488.436917] Lustre: Lustre: Build Version: 2.16.59_23_g5b5e124 [ 5488.625630] LNet: Added LNI 192.168.202.124@tcp [8/256/0/180] [ 5490.255507] Key type lgssc registered [ 5491.285952] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5497.327673] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5497.361526] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5498.753875] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5502.038674] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing set_default_debug -1 all [ 5508.069665] LustreError: 219025:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5508.081674] LustreError: 219025:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 5509.577945] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5510.025527] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5513.890660] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing set_default_debug -1 all [ 5519.092683] Lustre: DEBUG MARKER: oleg224-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 5524.569816] Lustre: DEBUG MARKER: oleg224-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 5533.838857] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5534.273183] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5539.692557] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing set_default_debug -1 all [ 5543.483157] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49666 to 0x280000401:49697) [ 5544.632889] Lustre: DEBUG MARKER: oleg224-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 5550.116046] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5551.726969] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5551.734341] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5551.742311] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5551.751347] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5551.754123] LustreError: 219044:0:(osp_precreate.c:1263:osp_precreate_thread()) cfs_fail_timeout id 2107 sleeping for 20000ms [ 5553.640926] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5553.642037] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5553.646275] Lustre: Skipped 1 previous similar message [ 5553.664662] Lustre: Skipped 1 previous similar message [ 5556.824056] Lustre: lustre-OST0000: Not available for connect from 192.168.202.24@tcp (stopping) [ 5558.756808] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5558.770374] Lustre: Skipped 1 previous similar message [ 5559.387845] LustreError: 221422:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 5559.656996] Lustre: server umount lustre-OST0000 complete [ 5571.831444] LustreError: 219044:0:(osp_precreate.c:1263:osp_precreate_thread()) cfs_fail_timeout id 2107 awake [ 5594.329191] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5594.607383] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5599.774600] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing set_default_debug -1 all [ 5605.002413] Lustre: DEBUG MARKER: oleg224-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 5606.730420] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid in FULL state after 0 sec [ 5608.946861] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5608.966898] LustreError: lustre-OST0000-osc-MDT0001: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5608.969255] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5610.116731] Lustre: DEBUG MARKER: oleg224-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff96af47fb4000.ost_server_uuid 50 [ 5611.558661] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff96af47fb4000.ost_server_uuid in FULL state after 0 sec [ 5616.828126] LustreError: 222742:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 5616.846115] LustreError: 222742:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 5617.043241] Lustre: server umount lustre-OST0000 complete [ 5622.752725] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 5622.764601] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5622.773849] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5629.408488] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5629.411412] Lustre: Skipped 3 previous similar messages [ 5630.432246] LustreError: 219044:0:(client.c:1370:ptlrpc_import_delay_req()) @@@ IMP_CLOSED req@00000000e7332f45 x1850523147886080/t0(0) o7->lustre-OST0000-osc-MDT0000@0@lo:28/4 lens 264/240 e 0 to 0 dl 0 ref 2 fl Rpc:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 5630.466822] LustreError: 222942:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup [ 5630.469549] LustreError: 222942:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 5630.543727] Lustre: server umount lustre-MDT0000 complete [ 5633.691961] LustreError: 219004:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1764796543 with bad export cookie 17927040448603297995 [ 5633.697997] LustreError: MGC192.168.202.124@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5633.702727] LustreError: 219004:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 5643.247083] LustreError: 223144:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [ 5643.258235] LustreError: 223144:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 5643.423470] Lustre: server umount lustre-MDT0001 complete [ 5651.088244] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing unload_modules_local [ 5653.174326] Key type lgssc unregistered [ 5653.398842] LNet: 223677:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5653.402448] LNetError: 223677:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5653.412542] LNet: Removed LNI 192.168.202.124@tcp [ 5654.088933] Key type .llcrypt unregistered [ 5654.090275] Key type ._llcrypt unregistered [ 5687.308510] Key type ._llcrypt registered [ 5687.310863] Key type .llcrypt registered [ 5687.402719] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing load_modules_local [ 5688.513942] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5688.551722] alg: No test for adler32 (adler32-zlib) [ 5689.530598] Lustre: Lustre: Build Version: 2.16.59_23_g5b5e124 [ 5689.717456] LNet: Added LNI 192.168.202.124@tcp [8/256/0/180] [ 5691.375804] Key type lgssc registered [ 5692.338356] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5700.457658] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5700.471341] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5701.832909] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5706.069468] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing set_default_debug -1 all [ 5712.354597] LustreError: 225386:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5712.394692] LustreError: 225386:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 5716.021225] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5716.386613] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5719.765229] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing set_default_debug -1 all [ 5722.640137] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 5729.524874] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5729.949228] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5736.342198] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing set_default_debug -1 all [ 5745.160475] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5745.235307] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5745.346348] Lustre: lustre-OST0001: new disk, initializing [ 5745.355894] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 5745.456321] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 5747.521899] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49699 to 0x280000401:49729) [ 5751.737136] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing set_default_debug -1 all [ 5753.959314] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 5753.974097] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 5754.055510] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 5761.468447] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 5767.537166] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 5784.739564] LustreError: 228744:0:(obd_class.h:479:obd_check_dev()) Device 28 not setup [ 5784.887411] Lustre: server umount lustre-MDT0000 complete [ 5785.060854] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5785.074122] LustreError: 225383:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5785.078144] Lustre: Skipped 2 previous similar messages [ 5785.096937] LustreError: 225383:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 5787.967995] LustreError: 225366:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1764796698 with bad export cookie 1576716809955686459 [ 5787.970707] LustreError: MGC192.168.202.124@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5787.998266] LustreError: 225366:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 5790.101562] LustreError: 228946:0:(obd_class.h:479:obd_check_dev()) Device 27 not setup [ 5790.106232] LustreError: 228946:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 5790.176476] LustreError: 225381:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5790.177368] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5790.183772] LustreError: 225381:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 5790.194056] Lustre: Skipped 2 previous similar messages [ 5790.309789] Lustre: server umount lustre-MDT0001 complete [ 5793.417802] LustreError: 229148:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 5793.419936] LustreError: 229148:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 5793.480448] Lustre: server umount lustre-OST0000 complete [ 5796.385502] LustreError: 229348:0:(obd_class.h:479:obd_check_dev()) Device 2 not setup [ 5796.390642] LustreError: 229348:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 5796.562108] Lustre: server umount lustre-OST0001 complete [ 5799.416676] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro [ 5799.489471] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5800.391165] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro [ 5800.500333] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5801.463510] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: ,errors=remount-ro [ 5801.559327] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5802.432854] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: ,errors=remount-ro [ 5802.496850] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5814.723574] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing load_modules_local [ 5824.527145] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5824.945045] LustreError: 230757:0:(ldlm_lib.c:1179:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5824.965977] LustreError: 230757:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 5825.061434] Lustre: mylustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5829.397962] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing set_default_debug -1 all [ 5830.124383] LustreError: 230756:0:(ldlm_lib.c:1179:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5830.134903] LustreError: 230756:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 5838.083218] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5838.586685] Lustre: mylustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5842.420787] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing set_default_debug -1 all [ 5845.657149] Lustre: 231868:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 5854.469991] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5854.908079] Lustre: mylustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5862.070673] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing set_default_debug -1 all [ 5863.095573] LustreError: 232224:0:(ldlm_lib.c:1179:target_handle_connect()) mylustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5863.101716] LustreError: 232224:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 5870.484069] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5870.587298] Lustre: mylustre-OST0000: new connection from mylustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49732 to 0x280000401:49761) [ 5876.547136] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing set_default_debug -1 all [ 5883.798637] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 5887.207888] Lustre: 233716:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 5901.799665] Lustre: mylustre-MDT0000-osp-MDT0001: Connection to mylustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5901.801558] Lustre: mylustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5901.811940] Lustre: Skipped 1 previous similar message [ 5901.827090] Lustre: Skipped 2 previous similar messages [ 5906.919098] Lustre: mylustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5906.927521] Lustre: Skipped 4 previous similar messages [ 5907.305712] LustreError: 234022:0:(obd_class.h:479:obd_check_dev()) Device 10 not setup [ 5907.307914] LustreError: 234022:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 5907.369713] Lustre: server umount mylustre-MDT0000 complete [ 5910.705308] LustreError: 231485:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1764796820 with bad export cookie 1576716809955689266 [ 5910.712441] LustreError: MGC192.168.202.124@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5910.722246] LustreError: 231485:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 5911.097969] Lustre: server umount mylustre-MDT0001 complete [ 5924.537628] LustreError: 234424:0:(obd_class.h:479:obd_check_dev()) Device 23 not setup [ 5924.540450] LustreError: 234424:0:(obd_class.h:479:obd_check_dev()) Skipped 15 previous similar messages [ 5924.592246] Lustre: server umount mylustre-OST0000 complete [ 5927.903105] Lustre: 224651:0:(client.c:2468:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1764796822/real 1764796822] req@ffff8d248d70bb80 x1850523358743808/t0(0) o400->mylustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1764796838 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 5927.953277] Lustre: mylustre-MDT0001-lwp-OST0001: Connection to mylustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5927.964546] Lustre: Skipped 2 previous similar messages [ 5928.326508] Lustre: server umount mylustre-OST0001 complete [ 5930.968894] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro [ 5931.026814] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5931.899080] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro [ 5931.958540] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5932.890435] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: ,errors=remount-ro [ 5932.965954] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5933.741866] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: ,errors=remount-ro [ 5933.795734] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5948.123481] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing load_modules_local [ 5957.551293] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5957.904859] LustreError: 236034:0:(ldlm_lib.c:1179:target_handle_connect()) tfs-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5957.925516] LustreError: 236034:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 5957.974345] Lustre: tfs-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5957.977431] Lustre: Skipped 1 previous similar message [ 5961.508779] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing set_default_debug -1 all [ 5969.346366] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5973.171985] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing set_default_debug -1 all [ 5976.449189] Lustre: 237144:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 5983.113487] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5985.522223] Lustre: tfs-OST0000: new connection from tfs-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49763 to 0x280000401:49793) [ 5988.558512] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing set_default_debug -1 all [ 5993.955353] LustreError: 237503:0:(ldlm_lib.c:1179:target_handle_connect()) tfs-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5993.963828] LustreError: 237503:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 6 previous similar messages [ 5997.151057] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 6004.054671] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing set_default_debug -1 all [ 6012.667719] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 6017.687862] Lustre: 238992:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 6031.224568] LustreError: 239239:0:(obd_class.h:479:obd_check_dev()) Device 10 not setup [ 6031.226890] LustreError: 239239:0:(obd_class.h:479:obd_check_dev()) Skipped 9 previous similar messages [ 6031.315732] Lustre: server umount tfs-MDT0000 complete [ 6031.355967] LustreError: tfs-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 6031.363425] Lustre: tfs-MDT0000-osp-MDT0001: Connection to tfs-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6035.226719] LustreError: 236009:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1764796945 with bad export cookie 1576716809955690932 [ 6035.231374] LustreError: MGC192.168.202.124@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6035.243975] LustreError: 236009:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 6035.597690] Lustre: server umount tfs-MDT0001 complete [ 6049.730179] Lustre: server umount tfs-OST0000 complete [ 6063.641483] LustreError: 239843:0:(obd_class.h:479:obd_check_dev()) Device 2 not setup [ 6063.644654] LustreError: 239843:0:(obd_class.h:479:obd_check_dev()) Skipped 19 previous similar messages [ 6063.905836] Lustre: server umount tfs-OST0001 complete [ 6066.939715] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro [ 6067.022533] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6068.011330] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro [ 6068.096504] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6069.198344] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: ,errors=remount-ro [ 6069.282286] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6070.420982] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: ,errors=remount-ro [ 6070.519837] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6082.304872] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing load_modules_local [ 6091.609932] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6092.114293] LustreError: 241257:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6092.129952] LustreError: 241257:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 5 previous similar messages [ 6092.226685] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6092.237557] Lustre: Skipped 3 previous similar messages [ 6095.866121] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing set_default_debug -1 all [ 6103.998977] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 6108.631793] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing set_default_debug -1 all [ 6111.872649] Lustre: 242368:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 6118.283618] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 6123.748370] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing set_default_debug -1 all [ 6131.681156] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 6136.756191] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49763 to 0x280000401:49825) [ 6137.722575] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing set_default_debug -1 all [ 6146.203761] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 6149.764230] Lustre: 244216:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 6161.024399] Lustre: DEBUG MARKER: == conf-sanity test 104a: Make sure user defined options are reflected in mount ========================================================== 16:24:30 (1764797070) [ 6177.771359] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6177.788460] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6177.798137] Lustre: Skipped 5 previous similar messages [ 6177.820369] Lustre: Skipped 3 previous similar messages [ 6182.862252] LustreError: 244589:0:(obd_class.h:479:obd_check_dev()) Device 10 not setup [ 6182.869343] LustreError: 244589:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 6182.882367] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6183.003383] Lustre: server umount lustre-MDT0000 complete [ 6186.333329] LustreError: 241233:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1764797096 with bad export cookie 1576716809955692374 [ 6186.338080] LustreError: MGC192.168.202.124@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6186.342557] LustreError: 241233:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 6188.006604] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 6188.017232] Lustre: Skipped 4 previous similar messages [ 6192.974542] Lustre: server umount lustre-MDT0001 complete [ 6211.039328] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 6211.210818] Lustre: server umount lustre-OST0000 complete [ 6220.663288] Lustre: server umount lustre-OST0001 complete [ 6225.265282] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing set_hostid [ 6232.972982] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing load_modules_local [ 6241.014288] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6247.514278] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6252.871335] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6257.569409] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6262.982637] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6263.051959] LDISKFS-fs (dm-0): Mount option "noacl" will be removed by 3.5 [ 6263.051959] Contact linux-ldiskfs@vger.kernel.org if you think we should keep it. [ 6263.051959] [ 6263.070754] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: acl,user_xattr,errors=remount-ro,noacl,no_mbcache,nodelalloc [ 6263.228614] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 6263.256862] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 6263.338709] Lustre: lustre-MDT0000: new disk, initializing [ 6263.427692] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6263.430984] Lustre: Skipped 3 previous similar messages [ 6263.455501] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 6266.567654] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing set_default_debug -1 all [ 6275.474173] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6275.529400] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 6275.605455] Lustre: 247382:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 6275.663391] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 6275.666789] Lustre: Skipped 1 previous similar message [ 6275.717342] Lustre: lustre-MDT0001: new disk, initializing [ 6275.845550] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 6275.852723] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 6278.567496] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing set_default_debug -1 all [ 6287.219595] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6287.276957] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 6287.521595] Lustre: lustre-OST0000: new disk, initializing [ 6287.530749] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 6289.139181] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 6289.155670] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 6289.260167] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 6292.193318] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing set_default_debug -1 all [ 6301.344344] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6301.402290] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 6301.502354] Lustre: lustre-OST0001: new disk, initializing [ 6301.505655] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 6303.565567] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 6303.573911] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 6303.655602] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 6306.284059] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing set_default_debug -1 all [ 6317.761737] Lustre: DEBUG MARKER: == conf-sanity test 104b: Mount uses last flock argument ========================================================== 16:27:07 (1764797227) [ 6330.211557] Lustre: DEBUG MARKER: == conf-sanity test 105: check file creation for ro and rw bind mnt pt ========================================================== 16:27:19 (1764797239) [ 6344.695992] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6344.704250] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6344.704581] Lustre: Skipped 4 previous similar messages [ 6344.707396] Lustre: Skipped 2 previous similar messages [ 6350.286044] LustreError: 250801:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 6350.299140] LustreError: 250801:0:(obd_class.h:479:obd_check_dev()) Skipped 25 previous similar messages [ 6350.400505] Lustre: server umount lustre-OST0000 complete [ 6354.926433] LustreError: 249056:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6354.951451] LustreError: 249056:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 7 previous similar messages [ 6359.895237] Lustre: server umount lustre-MDT0000 complete [ 6363.174182] LustreError: 247374:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1764797273 with bad export cookie 1576716809955693697 [ 6363.181877] LustreError: MGC192.168.202.124@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6363.190637] LustreError: 247374:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 5 previous similar messages [ 6365.160434] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 6365.163336] Lustre: Skipped 6 previous similar messages [ 6369.638727] Lustre: server umount lustre-MDT0001 complete [ 6380.093328] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing unload_modules_local [ 6396.908171] Lustre: server umount lustre-OST0001 complete [ 6402.136625] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing set_hostid [ 6415.257385] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing load_modules_local [ 6423.328727] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6429.447864] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6435.081580] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6440.792889] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6446.616246] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6446.677934] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6446.840236] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 6446.877534] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 6446.937634] Lustre: lustre-MDT0000: new disk, initializing [ 6447.033667] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 6450.633209] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing set_default_debug -1 all [ 6460.154759] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6460.240199] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 6460.373477] Lustre: 254293:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 6460.672635] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 6464.625775] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing set_default_debug -1 all [ 6473.307503] Lustre: DEBUG MARKER: oleg224-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 6476.713562] Lustre: DEBUG MARKER: oleg224-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 6484.146408] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6484.222775] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 6484.450865] Lustre: lustre-OST0000: new disk, initializing [ 6484.453259] Lustre: Skipped 1 previous similar message [ 6484.461365] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 6484.464117] Lustre: Skipped 2 previous similar messages [ 6486.537528] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 6486.554615] Lustre: Skipped 1 previous similar message [ 6486.567616] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 6486.710780] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 6490.070036] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing set_default_debug -1 all [ 6496.386191] Lustre: DEBUG MARKER: oleg224-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 6506.987663] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6506.991127] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 6507.002060] Lustre: Skipped 4 previous similar messages [ 6507.005605] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6507.012018] Lustre: Skipped 2 previous similar messages [ 6508.740388] Lustre: server umount lustre-OST0000 complete [ 6518.117950] Lustre: server umount lustre-MDT0000 complete [ 6521.905569] LustreError: 254283:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1764797432 with bad export cookie 1576716809955695167 [ 6521.915327] LustreError: MGC192.168.202.124@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6521.926453] LustreError: 254283:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 1 previous similar message [ 6522.470570] Lustre: server umount lustre-MDT0001 complete [ 6531.628527] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing unload_modules_local [ 6533.952922] Key type lgssc unregistered [ 6534.216796] LNet: 257671:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 6534.227408] LNetError: 257671:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 6534.252538] LNet: Removed LNI 192.168.202.124@tcp [ 6534.911324] Key type .llcrypt unregistered [ 6534.915673] Key type ._llcrypt unregistered [ 6561.048631] Key type ._llcrypt registered [ 6561.050253] Key type .llcrypt registered [ 6561.164658] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing load_modules_local [ 6562.371721] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 6562.397306] alg: No test for adler32 (adler32-zlib) [ 6563.524057] Lustre: Lustre: Build Version: 2.16.59_23_g5b5e124 [ 6563.749079] LNet: Added LNI 192.168.202.124@tcp [8/256/0/180] [ 6565.407190] Key type lgssc registered [ 6566.528757] Lustre: Echo OBD driver; http://www.lustre.org/ [ 6574.005039] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 6574.033380] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6575.512265] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6579.526229] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing set_default_debug -1 all [ 6584.817026] LustreError: 259114:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6584.831390] LustreError: 259114:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 6588.469510] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 6588.908665] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 6593.190761] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing set_default_debug -1 all [ 6599.050469] Lustre: DEBUG MARKER: oleg224-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 6602.771745] Lustre: DEBUG MARKER: oleg224-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 6609.513434] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 6609.901260] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 6615.105423] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing set_default_debug -1 all [ 6619.628440] Lustre: DEBUG MARKER: oleg224-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 6620.155063] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:3 to 0x280000401:33) [ 6620.456159] Lustre: Setting parameter lustre-OST0000.ost.unknown_param=50 in log lustre-OST0000 [ 6624.227970] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 6624.232045] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6624.251425] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6625.250817] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6625.256184] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6625.290850] Lustre: Skipped 1 previous similar message [ 6628.025454] LustreError: 261410:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 6628.227538] Lustre: server umount lustre-OST0000 complete [ 6634.976802] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 6634.979584] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6634.991520] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6637.424718] LustreError: 261617:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup [ 6637.429846] LustreError: 261617:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 6637.504841] Lustre: server umount lustre-MDT0000 complete [ 6640.613944] LustreError: 259114:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6640.641401] LustreError: 259114:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 6640.803244] LustreError: 259094:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1764797551 with bad export cookie 1856845494310780159 [ 6640.808429] LustreError: MGC192.168.202.124@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6640.813381] LustreError: 259094:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 6640.948242] LustreError: 261819:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [ 6640.953311] LustreError: 261819:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 6641.177789] Lustre: server umount lustre-MDT0001 complete [ 6649.700745] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing unload_modules_local [ 6652.218803] Key type lgssc unregistered [ 6652.554767] LNet: 262350:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 6652.573050] LNetError: 262350:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 6652.590722] LNet: Removed LNI 192.168.202.124@tcp [ 6653.399661] Key type .llcrypt unregistered [ 6653.403050] Key type ._llcrypt unregistered [ 6665.041369] Key type ._llcrypt registered [ 6665.043852] Key type .llcrypt registered [ 6665.145799] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing load_modules_local [ 6665.990802] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 6666.076921] alg: No test for adler32 (adler32-zlib) [ 6667.097070] Lustre: Lustre: Build Version: 2.16.59_23_g5b5e124 [ 6667.319223] LNet: Added LNI 192.168.202.124@tcp [8/256/0/180] [ 6668.976113] Key type lgssc registered [ 6669.900356] Lustre: Echo OBD driver; http://www.lustre.org/ [ 6677.631614] Lustre: lustre-OST0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 6677.652447] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 6693.151495] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 6698.271313] LustreError: 263504:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.202.124@tcp: failed processing log, type 4: rc = -110 [ 6724.960393] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 6729.641016] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing set_default_debug -1 all [ 6734.364682] Lustre: DEBUG MARKER: oleg224-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 6735.849048] LustreError: 264170:0:(obd_class.h:479:obd_check_dev()) Device 1 not setup [ 6735.977510] Lustre: server umount lustre-OST0000 complete [ 6747.768987] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing unload_modules_local [ 6749.946645] Key type lgssc unregistered [ 6750.170552] LNet: 265006:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 6750.178503] LNetError: 265006:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 6750.194595] LNet: Removed LNI 192.168.202.124@tcp [ 6750.910173] Key type .llcrypt unregistered [ 6750.912812] Key type ._llcrypt unregistered [ 6777.688880] Key type ._llcrypt registered [ 6777.690061] Key type .llcrypt registered [ 6777.748610] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing load_modules_local [ 6778.385267] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 6778.404858] alg: No test for adler32 (adler32-zlib) [ 6779.391605] Lustre: Lustre: Build Version: 2.16.59_23_g5b5e124 [ 6779.547715] LNet: Added LNI 192.168.202.124@tcp [8/256/0/180] [ 6781.199323] Key type lgssc registered [ 6782.041457] Lustre: Echo OBD driver; http://www.lustre.org/ [ 6791.079096] loop0: detected capacity change from 0 to 204800000 [ 6791.178549] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 6791.184429] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 6791.373604] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 6791.413088] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6792.216623] loop0: detected capacity change from 0 to 204800000 [ 6792.435392] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 6792.442615] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 6792.703382] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 6792.741270] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6793.514789] loop0: detected capacity change from 0 to 204800000 [ 6793.701536] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 6793.708659] blk_update_request: operation not supported error, dev loop0, sector 8224 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 6793.781648] blk_update_request: operation not supported error, dev loop0, sector 58264 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 6793.864344] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6794.558066] loop0: detected capacity change from 0 to 204800000 [ 6794.691976] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 6794.848274] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6795.645470] loop0: detected capacity change from 0 to 204800000 [ 6795.665821] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 6799.059919] loop0: detected capacity change from 0 to 204800000 [ 6799.087561] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 6802.452710] loop0: detected capacity change from 0 to 204800000 [ 6802.481547] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 6806.497643] loop0: detected capacity change from 0 to 204800000 [ 6806.539489] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 6809.782743] loop0: detected capacity change from 0 to 204800000 [ 6809.869094] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6809.924206] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 6809.937446] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6809.982669] Lustre: 268386:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 6809.992321] Lustre: 268386:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index 'nodemap' with [0x200000003:0x3:0x0]: rc = 0 [ 6810.002275] Lustre: 268386:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000003:0xc:0x0]: rc = 0 [ 6810.016453] Lustre: 268386:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000003:0xb:0x0]: rc = 0 [ 6810.028856] Lustre: 268386:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x1010000-MDT0000' with [0x200000005:0x2:0x0]: rc = 0 [ 6810.037483] Lustre: 268386:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x10000-MDT0000' with [0x200000005:0x1:0x0]: rc = 0 [ 6810.051358] Lustre: 268386:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x2010000-MDT0000' with [0x200000005:0x3:0x0]: rc = 0 [ 6810.066096] Lustre: 268386:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000003:0xa:0x0]: rc = 0 [ 6810.080189] Lustre: 268386:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x2020000' with [0x200000006:0x2020000:0x0]: rc = 0 [ 6810.098947] Lustre: 268386:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x1020000' with [0x200000006:0x1020000:0x0]: rc = 0 [ 6810.116187] Lustre: 268386:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x20000' with [0x200000006:0x20000:0x0]: rc = 0 [ 6810.128266] Lustre: 268386:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000006:0x2010000:0x0]: rc = 0 [ 6810.145159] Lustre: 268386:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000006:0x1010000:0x0]: rc = 0 [ 6810.156701] Lustre: 268386:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000006:0x10000:0x0]: rc = 0 [ 6815.319858] LustreError: 268654:0:(obd_class.h:479:obd_check_dev()) Device 2 not setup [ 6815.423231] Lustre: server umount MGS complete [ 6816.124335] loop0: detected capacity change from 0 to 204800000 [ 6816.217960] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6816.564980] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6816.614801] Lustre: lustre-MDT0000: Aborting client recovery [ 6817.458787] loop1: detected capacity change from 0 to 204800000 [ 6817.598676] LDISKFS-fs (loop1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6817.655298] LDISKFS-fs (loop1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 6817.705755] Lustre: 268823:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 6817.713460] Lustre: 268823:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0001: restore index '0x2010000' with [0x200000003:0x6:0x0]: rc = 0 [ 6817.720388] Lustre: 268823:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0001: restore index '0x10000' with [0x200000003:0x4:0x0]: rc = 0 [ 6817.727268] Lustre: 268823:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0001: restore index '0x2010000-MDT0001' with [0x200000005:0x3:0x0]: rc = 0 [ 6817.733798] Lustre: 268823:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0001: restore index '0x10000-MDT0001' with [0x200000005:0x1:0x0]: rc = 0 [ 6817.742831] Lustre: 268823:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0001: restore index '0x1010000-MDT0001' with [0x200000005:0x2:0x0]: rc = 0 [ 6817.752879] Lustre: 268823:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0001: restore index '0x1010000' with [0x200000003:0x5:0x0]: rc = 0 [ 6817.771789] Lustre: Found index 1 for lustre-MDT0001, updating log [ 6817.939565] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 6817.999147] Lustre: lustre-MDT0001: Aborting client recovery [ 6818.752054] loop2: detected capacity change from 0 to 204800000 [ 6818.830763] LDISKFS-fs (loop2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6818.901871] LDISKFS-fs (loop2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 6818.987544] Lustre: 268906:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 6819.002730] Lustre: 268906:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0000: restore index '0x1020000-OST0000' with [0x200000005:0x2:0x0]: rc = 0 [ 6819.007458] Lustre: 268906:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0000: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 6819.015577] Lustre: 268906:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0000: restore index '0x2020000-OST0000' with [0x200000005:0x3:0x0]: rc = 0 [ 6819.038733] Lustre: 268906:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0000: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 6819.051885] Lustre: 268906:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0000: restore index '0x20000-OST0000' with [0x200000005:0x1:0x0]: rc = 0 [ 6819.062182] Lustre: 268906:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0000: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 6819.158660] Lustre: Found index 0 for lustre-OST0000, updating log [ 6819.272710] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 6819.299761] Lustre: lustre-OST0000: Aborting recovery [ 6820.175167] loop3: detected capacity change from 0 to 204800000 [ 6820.276543] LDISKFS-fs (loop3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6820.387173] LDISKFS-fs (loop3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 6820.447991] Lustre: 268993:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 6820.453211] Lustre: 268993:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0001: restore index '0x2020000-OST0001' with [0x200000005:0x3:0x0]: rc = 0 [ 6820.458669] Lustre: 268993:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0001: restore index '0x20000-OST0001' with [0x200000005:0x1:0x0]: rc = 0 [ 6820.464409] Lustre: 268993:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0001: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 6820.470496] Lustre: 268993:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0001: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 6820.480825] Lustre: 268993:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0001: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 6820.488283] Lustre: 268993:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0001: restore index '0x1020000-OST0001' with [0x200000005:0x2:0x0]: rc = 0 [ 6820.501996] Lustre: Found index 1 for lustre-OST0001, updating log [ 6820.603565] Lustre: lustre-OST0001: Aborting recovery [ 6825.990490] LustreError: 268766:0:(lod_dev.c:510:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 10, retries 0, failed: rc = -108 [ 6826.985896] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 6826.988993] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 6827.084331] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000bd0 [ 6827.087886] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000bd0 [ 6828.007152] Lustre: lustre-OST0001: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:5 to 0x2c0000400:33) [ 6828.011163] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:5 to 0x280000400:33) [ 6829.322329] LustreError: 268734:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0000: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0000 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0000/identity_upcall, time 2589us: rc = -2 [ 6829.823494] LustreError: 269010:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0000: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0000 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0000/identity_upcall, time 2173us: rc = -2 [ 6829.835417] LustreError: 269010:0:(mdt_identity.c:115:mdt_identity_do_upcall()) Skipped 24 previous similar messages [ 6843.360214] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6843.372051] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6847.874859] LustreError: 269349:0:(obd_class.h:479:obd_check_dev()) Device 10 not setup [ 6847.879526] LustreError: 269349:0:(obd_class.h:479:obd_check_dev()) Skipped 1 previous similar message [ 6848.065344] Lustre: server umount lustre-MDT0000 complete [ 6848.485038] LustreError: 268730:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6848.492616] LustreError: 268730:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 5 previous similar messages [ 6848.956321] LustreError: 268717:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1764797759 with bad export cookie 18388787464482902756 [ 6848.962968] LustreError: MGC192.168.202.124@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6848.967893] LustreError: 268717:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 6853.600786] LustreError: 268732:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6853.608052] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6853.608972] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 6853.609081] Lustre: Skipped 3 previous similar messages [ 6853.624758] LustreError: 268732:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 6853.644318] Lustre: Skipped 4 previous similar messages [ 6858.720310] LustreError: 269010:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6858.720868] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 6858.757552] Lustre: Skipped 1 previous similar message [ 6863.327903] Lustre: lustre-MDT0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 6863.430293] LustreError: 269403:0:(obd_class.h:479:obd_check_dev()) Device 18 not setup [ 6863.432800] LustreError: 269403:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 6863.709343] Lustre: server umount lustre-MDT0001 complete [ 6870.656381] LustreError: 269456:0:(obd_class.h:479:obd_check_dev()) Device 23 not setup [ 6870.667414] LustreError: 269456:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 6870.759926] Lustre: server umount lustre-OST0000 complete [ 6877.628390] LustreError: 269507:0:(obd_class.h:479:obd_check_dev()) Device 2 not setup [ 6877.638149] LustreError: 269507:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 6877.814227] Lustre: server umount lustre-OST0001 complete [ 6885.401176] Lustre: DEBUG MARKER: == conf-sanity test 109a: test lctl clear_conf fsname ==== 16:36:34 (1764797794) [ 6909.551775] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing set_hostid [ 6917.439666] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing load_modules_local [ 6924.717892] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6929.743818] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6933.875847] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6938.046122] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6943.626927] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6943.680194] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6943.862221] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 6943.905838] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 6943.960385] Lustre: lustre-MDT0000: new disk, initializing [ 6944.050297] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6944.053494] Lustre: Skipped 1 previous similar message [ 6944.076637] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 6947.641275] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing set_default_debug -1 all [ 6956.429949] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6956.482138] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 6956.550602] Lustre: 273056:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 6956.594218] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 6956.597119] Lustre: Skipped 1 previous similar message [ 6956.641994] Lustre: lustre-MDT0001: new disk, initializing [ 6956.726432] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 6956.772765] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 6956.784449] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 6959.671215] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing set_default_debug -1 all [ 6965.786266] Lustre: DEBUG MARKER: oleg224-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 6968.997995] Lustre: DEBUG MARKER: oleg224-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 6974.785865] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6974.861839] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 6975.092563] Lustre: lustre-OST0000: new disk, initializing [ 6975.098763] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 6975.171757] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 6979.982515] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing set_default_debug -1 all [ 6982.209185] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 6982.224880] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 6982.318188] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 6986.260767] Lustre: DEBUG MARKER: oleg224-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 6990.973090] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [ 7002.855606] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [ 7013.708950] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [ 7022.146522] Lustre: Modifying parameter lustre-client.llite.max_read_ahead_mb=64 in log lustre-client [ 7069.671339] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 7069.672699] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7069.674966] LustreError: Skipped 1 previous similar message [ 7069.686842] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7069.693602] Lustre: Skipped 4 previous similar messages [ 7074.632115] LustreError: 276769:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 7074.636575] LustreError: 276769:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 7074.772673] Lustre: server umount lustre-OST0000 complete [ 7079.904604] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7079.904862] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7079.907759] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7079.913746] Lustre: Skipped 2 previous similar messages [ 7083.749407] Lustre: server umount lustre-MDT0000 complete [ 7085.024854] LustreError: 274250:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7085.032296] LustreError: 274250:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 4 previous similar messages [ 7086.532252] LustreError: 273844:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1764797996 with bad export cookie 18388787464482905458 [ 7086.536540] LustreError: MGC192.168.202.124@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7086.550349] LustreError: 273844:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 7087.015983] Lustre: server umount lustre-MDT0001 complete [ 7095.943840] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7101.071435] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing set_default_debug -1 all [ 7106.564181] LustreError: 278114:0:(obd_class.h:479:obd_check_dev()) Device 2 not setup [ 7106.567758] LustreError: 278114:0:(obd_class.h:479:obd_check_dev()) Skipped 17 previous similar messages [ 7106.636159] Lustre: server umount MGS complete [ 7117.831264] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7118.348293] LustreError: 278739:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7118.400986] LustreError: 278751:0:(qmt_pool.c:1523:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [ 7118.522674] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7122.495338] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing set_default_debug -1 all [ 7128.033756] LustreError: 278738:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7128.047917] LustreError: 278738:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 7130.048194] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 7133.515870] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing set_default_debug -1 all [ 7137.884493] Lustre: DEBUG MARKER: oleg224-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 7140.668434] Lustre: DEBUG MARKER: oleg224-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 7146.045813] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 7151.629909] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing set_default_debug -1 all [ 7156.505441] Lustre: DEBUG MARKER: oleg224-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 7191.525572] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 7191.539098] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7191.557067] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7191.560145] Lustre: Skipped 1 previous similar message [ 7193.575595] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 7195.899146] LustreError: 281420:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 7195.907913] LustreError: 281420:0:(obd_class.h:479:obd_check_dev()) Skipped 1 previous similar message [ 7196.058381] Lustre: server umount lustre-OST0000 complete [ 7202.273403] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7202.279179] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7202.296872] Lustre: Skipped 1 previous similar message [ 7205.528572] Lustre: server umount lustre-MDT0000 complete [ 7208.543614] LustreError: 279461:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1764798118 with bad export cookie 18388787464482907411 [ 7208.559881] LustreError: MGC192.168.202.124@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7208.912236] Lustre: server umount lustre-MDT0001 complete [ 7220.323236] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing unload_modules_local [ 7223.188296] Key type lgssc unregistered [ 7223.463609] LNet: 282355:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 7223.473251] LNetError: 282355:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 7223.485760] LNet: Removed LNI 192.168.202.124@tcp [ 7224.141150] Key type .llcrypt unregistered [ 7224.143853] Key type ._llcrypt unregistered [ 7258.451913] Key type ._llcrypt registered [ 7258.453852] Key type .llcrypt registered [ 7258.559193] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing set_hostid [ 7272.127174] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing load_modules_local [ 7273.221847] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 7273.314417] alg: No test for adler32 (adler32-zlib) [ 7274.441656] Lustre: Lustre: Build Version: 2.16.59_23_g5b5e124 [ 7274.673763] LNet: Added LNI 192.168.202.124@tcp [8/256/0/180] [ 7276.383226] Key type lgssc registered [ 7277.490641] Lustre: Echo OBD driver; http://www.lustre.org/ [ 7286.132164] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 7292.106717] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 7297.268708] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 7303.244614] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 7309.054618] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 7309.092925] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 7309.103645] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7310.377187] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 7310.417850] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 7310.486113] Lustre: lustre-MDT0000: new disk, initializing [ 7310.586501] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7310.606441] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 7313.984218] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing set_default_debug -1 all [ 7322.755625] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 7322.824431] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 7322.925262] Lustre: 286287:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 7322.992744] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 7322.995751] Lustre: Skipped 1 previous similar message [ 7323.063508] Lustre: lustre-MDT0001: new disk, initializing [ 7323.127202] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7323.172244] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 7323.181186] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 7326.432279] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing set_default_debug -1 all [ 7334.281977] Lustre: DEBUG MARKER: oleg224-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 7338.487976] Lustre: DEBUG MARKER: oleg224-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 7346.608631] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 7346.672878] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 7346.874266] Lustre: lustre-OST0000: new disk, initializing [ 7346.880630] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 7346.990838] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7352.410378] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing set_default_debug -1 all [ 7356.983675] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 7356.993356] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 7357.125754] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 7359.701804] Lustre: DEBUG MARKER: oleg224-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 7366.074944] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [ 7371.278654] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [ 7374.999076] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [ 7423.969244] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 7423.970428] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7423.984754] LustreError: Skipped 1 previous similar message [ 7424.008173] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7429.090141] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7429.095070] Lustre: Skipped 2 previous similar messages [ 7434.207981] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7434.214569] Lustre: Skipped 1 previous similar message [ 7438.303178] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 7438.371181] LustreError: 289612:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 7438.532370] Lustre: server umount lustre-OST0000 complete [ 7444.454941] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7444.456890] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7444.474531] Lustre: Skipped 2 previous similar messages [ 7444.493723] Lustre: Skipped 1 previous similar message [ 7447.662496] LustreError: 289814:0:(obd_class.h:479:obd_check_dev()) Device 22 not setup [ 7447.667089] LustreError: 289814:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 7449.703227] LustreError: 289814:0:(obd_class.h:479:obd_check_dev()) Device 8 not setup [ 7449.719322] LustreError: 289814:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 7449.789517] Lustre: server umount lustre-MDT0000 complete [ 7452.797189] LustreError: 287068:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1764798363 with bad export cookie 2122535395189836303 [ 7452.800659] LustreError: MGC192.168.202.124@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7452.804583] LustreError: 287068:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 7452.906166] LustreError: 290016:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 7452.909315] LustreError: 290016:0:(obd_class.h:479:obd_check_dev()) Skipped 1 previous similar message [ 7453.084347] Lustre: server umount lustre-MDT0001 complete [ 7460.507245] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7463.370226] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing set_default_debug -1 all [ 7467.075765] LustreError: 290958:0:(obd_class.h:479:obd_check_dev()) Device 2 not setup [ 7467.083186] LustreError: 290958:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 7467.148437] Lustre: server umount MGS complete [ 7476.822438] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7477.340179] LustreError: 291583:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7477.358433] LustreError: 291583:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 7477.367401] LustreError: 291595:0:(qmt_pool.c:1523:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [ 7477.447405] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7481.394777] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing set_default_debug -1 all [ 7482.850382] LustreError: 291583:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7486.948358] LustreError: 291583:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7490.461757] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 7490.710335] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7494.618853] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing set_default_debug -1 all [ 7500.018766] Lustre: DEBUG MARKER: oleg224-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 7503.960863] Lustre: DEBUG MARKER: oleg224-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 7511.979289] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 7512.404659] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7518.986559] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing set_default_debug -1 all [ 7525.243717] Lustre: DEBUG MARKER: oleg224-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 7556.581045] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 7556.600189] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7556.622213] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7558.627406] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7561.487113] LustreError: 294241:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 7561.490216] LustreError: 294241:0:(obd_class.h:479:obd_check_dev()) Skipped 1 previous similar message [ 7561.686459] Lustre: server umount lustre-OST0000 complete [ 7567.842775] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7567.855782] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7567.866660] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7567.869717] Lustre: Skipped 2 previous similar messages [ 7571.324123] Lustre: server umount lustre-MDT0000 complete [ 7573.997134] LustreError: 291578:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7574.041604] LustreError: 291578:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 7575.719313] LustreError: 291562:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1764798485 with bad export cookie 2122535395189838109 [ 7575.724264] LustreError: MGC192.168.202.124@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7575.746574] LustreError: 291562:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 7576.183171] Lustre: server umount lustre-MDT0001 complete [ 7589.329804] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing unload_modules_local [ 7591.982745] Key type lgssc unregistered [ 7592.263109] LNet: 295175:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 7592.280336] LNetError: 295175:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 7593.317488] LNet: Removed LNI 192.168.202.124@tcp [ 7594.137396] Key type .llcrypt unregistered [ 7594.139157] Key type ._llcrypt unregistered [ 7611.536093] Key type ._llcrypt registered [ 7611.537806] Key type .llcrypt registered [ 7611.621549] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing set_hostid [ 7622.857321] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing load_modules_local [ 7623.502771] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 7623.571186] alg: No test for adler32 (adler32-zlib) [ 7624.578757] Lustre: Lustre: Build Version: 2.16.59_23_g5b5e124 [ 7624.784603] LNet: Added LNI 192.168.202.124@tcp [8/256/0/180] [ 7626.440483] Key type lgssc registered [ 7627.251487] Lustre: Echo OBD driver; http://www.lustre.org/ [ 7633.809655] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 7638.527340] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 7645.912549] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing load_modules_local [ 7653.136949] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 7658.563910] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 7658.604138] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 7658.614045] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7659.823843] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 7659.849489] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 7659.908924] Lustre: lustre-MDT0000: new disk, initializing [ 7659.988417] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7660.004440] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 7663.845761] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing set_default_debug -1 all [ 7673.730496] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 7673.824154] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 7674.233777] Lustre: lustre-OST0000: new disk, initializing [ 7674.248515] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 7674.255858] Lustre: Skipped 1 previous similar message [ 7674.374736] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7679.527933] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 7679.546066] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 7679.599645] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 7679.721211] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing set_default_debug -1 all [ 8549.858617] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8549.881946] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8550.934634] LustreError: 300230:0:(obd_class.h:479:obd_check_dev()) Device 12 not setup [ 8551.247665] Lustre: server umount lustre-MDT0000 complete [ 8554.722034] LustreError: 298363:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1764799464 with bad export cookie 6893871245395296961 [ 8554.731772] LustreError: 298363:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 8554.732052] LustreError: MGC192.168.202.124@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8554.764200] LustreError: 300433:0:(obd_class.h:479:obd_check_dev()) Device 2 not setup [ 8554.767449] LustreError: 300433:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 8554.954852] Lustre: server umount lustre-OST0000 complete [ 8578.380703] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing set_hostid [ 8587.078199] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing load_modules_local [ 8596.227762] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 8603.562531] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 8611.454892] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 8619.000781] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 8625.688806] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 8625.765324] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8626.090651] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 8626.141805] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 8626.243669] Lustre: lustre-MDT0000: new disk, initializing [ 8626.382992] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8626.424827] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 8630.179817] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing set_default_debug -1 all [ 8641.015517] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 8641.079267] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 8641.134929] Lustre: 303477:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 8641.180229] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 8641.183894] Lustre: Skipped 1 previous similar message [ 8641.250132] Lustre: lustre-MDT0001: new disk, initializing [ 8641.333638] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8641.389174] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 8641.402097] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 8644.752679] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing set_default_debug -1 all [ 8652.081587] Lustre: DEBUG MARKER: oleg224-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 8655.363840] Lustre: DEBUG MARKER: oleg224-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 8661.986972] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 8662.062532] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 8662.255505] Lustre: lustre-OST0000: new disk, initializing [ 8662.258809] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 8662.347825] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8667.277218] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing set_default_debug -1 all [ 8671.798167] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 8671.803559] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 8671.900379] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 8674.082597] Lustre: DEBUG MARKER: oleg224-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 8677.926086] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 8678.099611] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 8681.692756] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 8681.924388] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 8687.077423] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 8687.082844] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8687.097200] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8687.599683] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 8689.498102] LustreError: 306416:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 8689.500887] LustreError: 306416:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 8689.661459] Lustre: server umount lustre-OST0000 complete [ 8697.824133] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8697.828661] Lustre: Skipped 1 previous similar message [ 8697.831083] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8697.832304] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8697.832308] Lustre: Skipped 1 previous similar message [ 8699.627554] LustreError: 306617:0:(obd_class.h:479:obd_check_dev()) Device 22 not setup [ 8699.645715] LustreError: 306617:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 8699.844447] Lustre: server umount lustre-MDT0000 complete [ 8703.871170] LustreError: 303471:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1764799614 with bad export cookie 6893871245398237808 [ 8703.875531] LustreError: MGC192.168.202.124@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8703.888775] LustreError: 303471:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 1 previous similar message [ 8704.040181] LustreError: 306819:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 8704.044399] LustreError: 306819:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 8704.354644] Lustre: server umount lustre-MDT0001 complete [ 8707.916021] Lustre: DEBUG MARKER: SKIP: conf-sanity test_111 skipping SLOW test 111 [ 8710.423448] Lustre: DEBUG MARKER: == conf-sanity test 112a: mount OST with no_create option ========================================================== 17:06:59 (1764799619) [ 8717.149269] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8717.575310] LustreError: 307393:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8717.602497] LustreError: 307393:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 8717.672911] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8721.795776] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing set_default_debug -1 all [ 8722.919179] LustreError: 307393:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8727.017823] LustreError: 307393:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8729.711752] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 8730.086147] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8733.504784] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing set_default_debug -1 all [ 8738.101762] Lustre: DEBUG MARKER: oleg224-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 8741.236204] Lustre: DEBUG MARKER: oleg224-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 8747.640684] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 8747.904286] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8752.714799] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing set_default_debug -1 all [ 8757.649772] Lustre: DEBUG MARKER: oleg224-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 8764.147710] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 8764.217526] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 8764.400848] Lustre: lustre-OST0001: new disk, initializing [ 8764.406458] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 8764.486786] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 8770.172146] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing set_default_debug -1 all [ 8771.680132] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 8771.691412] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 8771.753115] LustreError: 310332:0:(osp_precreate.c:969:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [ 8771.765610] LustreError: 310332:0:(osp_precreate.c:969:osp_precreate_cleanup_orphans()) Skipped 1 previous similar message [ 8772.773586] LustreError: 310323:0:(osp_precreate.c:969:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [ 8773.799517] LustreError: 310323:0:(osp_precreate.c:969:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [ 8773.819232] LustreError: 310323:0:(osp_precreate.c:969:osp_precreate_cleanup_orphans()) Skipped 1 previous similar message [ 8775.906897] LustreError: 310323:0:(osp_precreate.c:969:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [ 8775.917779] LustreError: 310323:0:(osp_precreate.c:969:osp_precreate_cleanup_orphans()) Skipped 3 previous similar messages [ 8778.783896] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 8778.990258] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 8780.002421] LustreError: 310323:0:(osp_precreate.c:969:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [ 8780.009712] LustreError: 310323:0:(osp_precreate.c:969:osp_precreate_cleanup_orphans()) Skipped 7 previous similar messages [ 8782.411053] Lustre: DEBUG MARKER: oleg224-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff96af430be000.ost_server_uuid 50 [ 8783.916995] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff96af430be000.ost_server_uuid in FULL state after 0 sec [ 8787.545316] Lustre: DEBUG MARKER: oleg224-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff96af430be000.ost_server_uuid 50 [ 8788.206652] LustreError: 310323:0:(osp_precreate.c:969:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [ 8788.215733] LustreError: 310323:0:(osp_precreate.c:969:osp_precreate_cleanup_orphans()) Skipped 15 previous similar messages [ 8789.223333] Lustre: DEBUG MARKER: osc.lustre-OST0001-osc-ffff96af430be000.ost_server_uuid in FULL state after 0 sec [ 8791.367938] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 8805.990714] Lustre: lustre-OST0001: Not available for connect from 192.168.202.24@tcp (stopping) [ 8805.998415] Lustre: Skipped 1 previous similar message [ 8806.882575] LustreError: lustre-OST0001-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 8806.899671] Lustre: lustre-OST0001-osc-MDT0001: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8810.551707] LustreError: 311204:0:(obd_class.h:479:obd_check_dev()) Device 25 not setup [ 8810.558985] LustreError: 311204:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 8810.640237] Lustre: server umount lustre-OST0001 complete [ 8811.091189] LustreError: 308989:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 192.168.202.24@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8816.608289] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 8816.612229] LustreError: Skipped 2 previous similar messages [ 8816.620423] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8816.638775] Lustre: Skipped 1 previous similar message [ 8816.650318] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8816.663496] Lustre: Skipped 4 previous similar messages [ 8818.154566] LustreError: 311205:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8818.171739] LustreError: 311205:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 8822.213760] Lustre: server umount lustre-OST0000 complete [ 8827.360418] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8827.368121] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8827.382519] Lustre: Skipped 1 previous similar message [ 8827.390759] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8827.396827] Lustre: Skipped 2 previous similar messages [ 8832.248436] LustreError: 311608:0:(obd_class.h:479:obd_check_dev()) Device 28 not setup [ 8832.256907] LustreError: 311608:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 8832.456174] Lustre: server umount lustre-MDT0000 complete [ 8833.504281] LustreError: 307389:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8833.525354] LustreError: 307389:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 8835.867598] LustreError: 307374:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1764799746 with bad export cookie 6893871245398238690 [ 8835.873410] LustreError: MGC192.168.202.124@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8836.413181] Lustre: server umount lustre-MDT0001 complete [ 8846.532900] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing unload_modules_local [ 8849.162836] Key type lgssc unregistered [ 8849.379820] LNet: 312341:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8849.386988] LNetError: 312341:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8849.399183] LNet: Removed LNI 192.168.202.124@tcp [ 8850.087630] Key type .llcrypt unregistered [ 8850.092448] Key type ._llcrypt unregistered [ 8871.278122] Key type ._llcrypt registered [ 8871.279591] Key type .llcrypt registered [ 8871.371245] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing load_modules_local [ 8872.210881] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8872.218796] alg: No test for adler32 (adler32-zlib) [ 8873.404830] Lustre: Lustre: Build Version: 2.16.59_23_g5b5e124 [ 8873.644752] LNet: Added LNI 192.168.202.124@tcp [8/256/0/180] [ 8875.343189] Key type lgssc registered [ 8876.515625] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8884.286524] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 8884.321851] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8885.833323] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8889.545816] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing set_default_debug -1 all [ 8895.305249] Lustre: DEBUG MARKER: oleg224-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 8895.974493] LustreError: 313747:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8896.010378] LustreError: 313747:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 8901.089507] LustreError: 313747:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8902.126369] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 8902.442620] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8906.191511] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing set_default_debug -1 all [ 8915.048942] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 8915.485320] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8920.841994] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing set_default_debug -1 all [ 8925.521792] Lustre: DEBUG MARKER: oleg224-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 8926.758260] LustreError: 315299:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8930.858258] LustreError: 315299:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8930.865326] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:65) [ 8930.869066] LustreError: 315299:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 8931.145348] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 8931.327344] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 8936.334984] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing set_default_debug -1 all [ 8936.450886] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:33) [ 8941.542222] Lustre: DEBUG MARKER: oleg224-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid [ 8947.850739] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing wait_import_state (FULL|IDLE) os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [ 8948.087149] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 9059.299442] LustreError: lustre-OST0001-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 9059.303091] Lustre: lustre-OST0001-osc-MDT0000: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9059.320734] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 9059.324031] Lustre: Skipped 1 previous similar message [ 9061.117078] LustreError: 317178:0:(obd_class.h:479:obd_check_dev()) Device 27 not setup [ 9061.231981] Lustre: server umount lustre-OST0001 complete [ 9064.419203] LustreError: 316537:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9064.437619] LustreError: 316537:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 9069.026861] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 9069.046592] LustreError: Skipped 1 previous similar message [ 9069.052737] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9069.075586] Lustre: Skipped 1 previous similar message [ 9069.086453] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9072.571182] LustreError: 317379:0:(obd_class.h:479:obd_check_dev()) Device 23 not setup [ 9072.582605] LustreError: 317379:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 9072.808209] Lustre: server umount lustre-OST0000 complete [ 9076.704351] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 9076.716758] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9076.728308] Lustre: Skipped 1 previous similar message [ 9076.736524] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9076.745503] Lustre: Skipped 1 previous similar message [ 9079.777395] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9079.777815] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9079.792017] Lustre: Skipped 1 previous similar message [ 9082.471726] LustreError: 317581:0:(obd_class.h:479:obd_check_dev()) Device 10 not setup [ 9082.482971] LustreError: 317581:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 9082.577196] Lustre: server umount lustre-MDT0000 complete [ 9084.914239] LustreError: 314880:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9084.933215] LustreError: 314880:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 9086.239713] LustreError: 316209:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1764799996 with bad export cookie 15830325379162000253 [ 9086.248508] LustreError: MGC192.168.202.124@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9086.261154] LustreError: 316209:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 9086.411806] LustreError: 317782:0:(obd_class.h:479:obd_check_dev()) Device 18 not setup [ 9086.414453] LustreError: 317782:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 9086.674830] Lustre: server umount lustre-MDT0001 complete [ 9097.114168] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing unload_modules_local [ 9099.687363] Key type lgssc unregistered [ 9099.954029] LNet: 318314:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 9099.966247] LNetError: 318314:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 9099.977734] LNet: Removed LNI 192.168.202.124@tcp [ 9100.681084] Key type .llcrypt unregistered [ 9100.683392] Key type ._llcrypt unregistered [ 9139.949152] Key type ._llcrypt registered [ 9139.950390] Key type .llcrypt registered [ 9140.056920] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing load_modules_local [ 9140.669130] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 9140.682629] alg: No test for adler32 (adler32-zlib) [ 9141.718443] Lustre: Lustre: Build Version: 2.16.59_23_g5b5e124 [ 9141.908227] LNet: Added LNI 192.168.202.124@tcp [8/256/0/180] [ 9143.559176] Key type lgssc registered [ 9144.588513] Lustre: Echo OBD driver; http://www.lustre.org/ [ 9153.377627] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 9153.402514] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9154.935905] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9158.629129] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing set_default_debug -1 all [ 9165.283380] LustreError: 320564:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9165.305053] LustreError: 320564:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 9167.090311] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 9167.460783] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9170.897685] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing set_default_debug -1 all [ 9173.628699] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 9179.468782] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 9179.780937] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9180.784334] LustreError: 322109:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9180.794163] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:97) [ 9184.831697] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing set_default_debug -1 all [ 9186.286691] LustreError: 322109:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9191.398622] LustreError: 322109:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9192.341885] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 9192.574058] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 9197.177826] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing set_default_debug -1 all [ 9198.073290] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:65) [ 9206.208852] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 9212.091070] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 9228.770872] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9228.773814] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 9228.773991] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9228.786214] Lustre: Skipped 3 previous similar messages [ 9233.374400] LustreError: 323981:0:(obd_class.h:479:obd_check_dev()) Device 10 not setup [ 9233.587051] Lustre: server umount lustre-MDT0000 complete [ 9233.889702] LustreError: 321279:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9233.919089] LustreError: 321279:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 5 previous similar messages [ 9237.069598] LustreError: 323582:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1764800147 with bad export cookie 14574273842326219915 [ 9237.075354] LustreError: MGC192.168.202.124@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9237.081603] LustreError: 323582:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 9237.216495] LustreError: 324182:0:(obd_class.h:479:obd_check_dev()) Device 18 not setup [ 9237.224744] LustreError: 324182:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 9237.441952] Lustre: server umount lustre-MDT0001 complete [ 9251.608853] LustreError: 324383:0:(obd_class.h:479:obd_check_dev()) Device 23 not setup [ 9251.613332] LustreError: 324383:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 9251.721534] Lustre: server umount lustre-OST0000 complete [ 9265.223870] LustreError: 324584:0:(obd_class.h:479:obd_check_dev()) Device 2 not setup [ 9265.227068] LustreError: 324584:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 9265.371722] Lustre: server umount lustre-OST0001 complete [ 9280.360333] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing load_modules_local [ 9289.121274] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9289.465349] LustreError: 325764:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9289.544591] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9292.896304] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing set_default_debug -1 all [ 9300.549226] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 9300.935548] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9304.328502] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing set_default_debug -1 all [ 9306.964733] Lustre: 326873:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 9312.839668] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 9314.185227] LustreError: 327229:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9314.202336] LustreError: 327229:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 9314.207497] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:129) [ 9317.777681] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing set_default_debug -1 all [ 9324.010235] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 9324.262538] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 9324.269956] Lustre: Skipped 1 previous similar message [ 9326.229238] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:97) [ 9328.812509] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing set_default_debug -1 all [ 9335.487604] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 9339.223597] Lustre: 328720:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 9347.044707] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 9347.048619] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9347.061072] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9347.064183] Lustre: Skipped 3 previous similar messages [ 9348.068951] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9348.069152] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9348.079387] Lustre: Skipped 2 previous similar messages [ 9352.739475] LustreError: 328825:0:(obd_class.h:479:obd_check_dev()) Device 10 not setup [ 9352.742938] LustreError: 328825:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 9352.818127] Lustre: server umount lustre-MDT0000 complete [ 9353.185585] LustreError: 326480:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9353.198305] LustreError: 326480:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 4 previous similar messages [ 9355.412931] LustreError: 325743:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1764800265 with bad export cookie 14574273842326221609 [ 9355.415262] LustreError: MGC192.168.202.124@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9355.424325] LustreError: 325743:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 1 previous similar message [ 9355.698557] Lustre: server umount lustre-MDT0001 complete [ 9368.605309] LustreError: 329230:0:(obd_class.h:479:obd_check_dev()) Device 23 not setup [ 9368.662388] LustreError: 329230:0:(obd_class.h:479:obd_check_dev()) Skipped 15 previous similar messages [ 9368.689034] Lustre: server umount lustre-OST0000 complete [ 9381.613484] Lustre: server umount lustre-OST0001 complete [ 9383.611364] Lustre: DEBUG MARKER: SKIP: conf-sanity test_114 skipping SLOW test 114 [ 9384.487295] Lustre: DEBUG MARKER: SKIP: conf-sanity test_115 skipping excluded test 115 [ 9385.352777] Lustre: DEBUG MARKER: == conf-sanity test 116: big size MDT support ============ 17:18:15 (1764800295) [ 9397.756896] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing load_modules_local [ 9403.235767] loop0: detected capacity change from 0 to 1099511627776 [ 9403.527312] SGI XFS with ACLs, security attributes, quota, no debug enabled [ 9403.530575] *** VALIDATE xfs *** [ 9403.549893] XFS (loop0): Mounting V5 Filesystem [ 9403.583864] XFS (loop0): Ending clean mount [ 9404.610521] loop1: detected capacity change from 0 to 18691697672192 [ 9445.749681] LDISKFS-fs (loop1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 9457.868157] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing set_hostid [ 9465.285335] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing load_modules_local [ 9472.079794] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 9477.540964] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 9483.753036] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 9489.733031] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 9496.165556] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 9496.212144] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9496.378961] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 9496.407126] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 9496.514984] Lustre: lustre-MDT0000: new disk, initializing [ 9496.624991] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9496.652885] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 9500.066357] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing set_default_debug -1 all [ 9508.616858] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 9508.680236] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 9508.736822] Lustre: 333807:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 9508.787580] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 9508.793371] Lustre: Skipped 1 previous similar message [ 9508.843272] Lustre: lustre-MDT0001: new disk, initializing [ 9508.963764] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 9508.972677] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 9511.802325] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing set_default_debug -1 all [ 9517.587598] Lustre: DEBUG MARKER: oleg224-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 9520.594915] Lustre: DEBUG MARKER: oleg224-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 9525.688414] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 9525.748785] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 9525.935986] Lustre: lustre-OST0000: new disk, initializing [ 9525.943472] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 9527.270912] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 9527.277449] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 9527.371049] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 9530.185619] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing set_default_debug -1 all [ 9535.833517] Lustre: DEBUG MARKER: oleg224-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 9538.791780] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 9538.926190] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 9541.298330] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 9541.439295] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 9547.746353] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 9547.748080] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9547.754086] LustreError: Skipped 1 previous similar message [ 9547.756160] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9547.770861] Lustre: Skipped 1 previous similar message [ 9547.788840] Lustre: Skipped 4 previous similar messages [ 9548.718182] LustreError: 336746:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 9548.722902] LustreError: 336746:0:(obd_class.h:479:obd_check_dev()) Skipped 9 previous similar messages [ 9548.845193] Lustre: server umount lustre-OST0000 complete [ 9552.870408] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9552.871319] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9552.882512] Lustre: Skipped 1 previous similar message [ 9552.893351] Lustre: Skipped 1 previous similar message [ 9557.922220] Lustre: server umount lustre-MDT0000 complete [ 9557.987700] LustreError: 333818:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9557.997985] LustreError: 333818:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 9560.545681] LustreError: 336747:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1764800470 with bad export cookie 14574273842326223170 [ 9560.549668] LustreError: MGC192.168.202.124@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9560.562568] LustreError: 336747:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 9560.860067] Lustre: server umount lustre-MDT0001 complete [ 9567.076831] XFS (loop0): Unmounting Filesystem [ 9570.677866] Lustre: DEBUG MARKER: == conf-sanity test 117: lctl get_param return errors properly ========================================================== 17:21:20 (1764800480) [ 9576.611920] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9577.153453] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9577.157718] Lustre: Skipped 2 previous similar messages [ 9580.359566] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing set_default_debug -1 all [ 9588.699288] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 9592.654459] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing set_default_debug -1 all [ 9598.229420] Lustre: DEBUG MARKER: oleg224-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 9601.972536] Lustre: DEBUG MARKER: oleg224-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 9607.983697] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 9613.600429] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing set_default_debug -1 all [ 9618.243983] Lustre: DEBUG MARKER: oleg224-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 9632.224434] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 9632.233544] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9632.239673] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9632.249666] Lustre: Skipped 1 previous similar message [ 9632.263761] Lustre: Skipped 1 previous similar message [ 9637.264058] LustreError: 340311:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 9637.276570] LustreError: 340311:0:(obd_class.h:479:obd_check_dev()) Skipped 17 previous similar messages [ 9637.482990] Lustre: server umount lustre-OST0000 complete [ 9652.704512] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9652.717187] Lustre: Skipped 5 previous similar messages [ 9655.264146] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 9655.448124] Lustre: server umount lustre-MDT0000 complete [ 9658.432697] LustreError: 337948:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1764800568 with bad export cookie 14574273842326224052 [ 9658.435802] LustreError: MGC192.168.202.124@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9658.438775] LustreError: 337948:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 9658.817802] Lustre: server umount lustre-MDT0001 complete [ 9667.167461] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing unload_modules_local [ 9669.452708] Key type lgssc unregistered [ 9669.695873] LNet: 341247:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 9669.704522] LNetError: 341247:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 9669.720538] LNet: Removed LNI 192.168.202.124@tcp [ 9670.439148] Key type .llcrypt unregistered [ 9670.441655] Key type ._llcrypt unregistered [ 9691.200628] Key type ._llcrypt registered [ 9691.202429] Key type .llcrypt registered [ 9691.311927] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing load_modules_local [ 9692.409932] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 9692.430296] alg: No test for adler32 (adler32-zlib) [ 9693.507243] Lustre: Lustre: Build Version: 2.16.59_23_g5b5e124 [ 9693.706640] LNet: Added LNI 192.168.202.124@tcp [8/256/0/180] [ 9695.409997] Key type lgssc registered [ 9696.343844] Lustre: Echo OBD driver; http://www.lustre.org/ [ 9703.674660] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 9703.692532] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9705.097251] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9708.134222] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing set_default_debug -1 all [ 9715.185576] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 9715.454305] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9718.442639] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing set_default_debug -1 all [ 9722.997466] Lustre: DEBUG MARKER: oleg224-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 9726.438037] Lustre: DEBUG MARKER: oleg224-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 9731.662986] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 9731.747776] LustreError: 344375:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9731.757476] LustreError: 344375:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 9731.947807] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9736.893540] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing set_default_debug -1 all [ 9742.171143] Lustre: DEBUG MARKER: oleg224-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 9750.120230] Lustre: lustre-MDT0001: Not available for connect from 192.168.202.24@tcp (stopping) [ 9750.498491] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9755.206712] Lustre: lustre-MDT0001: Not available for connect from 192.168.202.24@tcp (stopping) [ 9755.217510] Lustre: Skipped 2 previous similar messages [ 9755.468093] LustreError: 345070:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [ 9755.619924] Lustre: server umount lustre-MDT0001 complete [ 9755.625142] LustreError: 342771:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9755.655222] LustreError: 342771:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 9758.675550] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 9760.329747] LustreError: 342771:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.202.24@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9765.261112] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 9765.314884] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [ 9765.317918] Lustre: Found index 1 for lustre-MDT0001, updating log [ 9765.348214] Lustre: 342765:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 9765.451409] LustreError: 343562:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.202.24@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9765.468417] LustreError: 343562:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 9765.648292] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9768.754399] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing set_default_debug -1 all [ 9770.478774] LustreError: 346003:0:(obd_mount.c:188:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [ 9770.496328] LustreError: 346003:0:(tgt_mount.c:669:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [ 9770.511345] LustreError: 342768:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [ 9770.552406] LustreError: 346007:0:(obd_config.c:1997:class_config_llog_handler()) MGC192.168.202.124@tcp: cfg command failed: rc = -17 [ 9770.556765] Lustre: cmd=cf001 0:lustre-MDT0001-osp-MDT0000 1:osp 2:lustre-MDT0000-mdtlov_UUID [ 9770.556765] [ 9770.995512] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [10072.662701] Lustre: lustre-MDT0001: Not available for connect from 192.168.202.24@tcp (stopping) [10073.061073] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10073.066278] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [10073.069715] Lustre: Skipped 1 previous similar message [10077.768396] Lustre: lustre-MDT0001: Not available for connect from 192.168.202.24@tcp (stopping) [10077.773663] Lustre: Skipped 2 previous similar messages [10078.487233] LustreError: 354737:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [10078.490906] LustreError: 354737:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [10078.553337] Lustre: server umount lustre-MDT0001 complete [10080.590953] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [10082.889439] LustreError: 343562:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.202.24@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [10086.003617] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [10086.057703] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [10086.060955] Lustre: Found index 1 for lustre-MDT0001, updating log [10086.090492] Lustre: 342764:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [10086.344617] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10089.164709] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing set_default_debug -1 all [10091.497440] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [10096.125809] LustreError: 355816:0:(obd_mount.c:188:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [10096.135587] LustreError: 355816:0:(tgt_mount.c:669:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [10096.141819] LustreError: 342768:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [10096.145613] LustreError: 342768:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [10096.199410] LustreError: 355819:0:(obd_config.c:689:class_setup()) Device 8 already setup (type osp) [10096.203418] LustreError: 355819:0:(obd_config.c:1997:class_config_llog_handler()) MGC192.168.202.124@tcp: cfg command failed: rc = -17 [10096.208051] Lustre: cmd=cf003 0:lustre-MDT0001-osp-MDT0000 1:lustre-MDT0001_UUID 2:192.168.202.124@tcp [10096.208051] [10395.247020] Lustre: lustre-MDT0001: Not available for connect from 192.168.202.24@tcp (stopping) [10395.258392] Lustre: Skipped 2 previous similar messages [10398.695224] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10398.696349] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [10398.704872] Lustre: Skipped 2 previous similar messages [10400.135259] LustreError: 364955:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [10400.138558] LustreError: 364955:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [10400.196816] Lustre: server umount lustre-MDT0001 complete [10400.325407] LustreError: 343562:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.202.24@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [10400.335505] LustreError: 343562:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [10402.110341] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [10408.539192] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [10408.650259] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [10408.653058] Lustre: Found index 1 for lustre-MDT0001, updating log [10408.717107] Lustre: 342765:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [10408.928104] Lustre: lustre-MDT0001: Not available for connect from 0@lo (not set up) [10408.939025] Lustre: Skipped 3 previous similar messages [10409.179841] Lustre: lustre-MDT0001: Imperative Recovery enabled, recovery window shrunk from 60-180 down to 60-180 [10412.698712] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing set_default_debug -1 all [10414.566527] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [10418.681783] LustreError: 365989:0:(obd_mount.c:188:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [10418.685844] LustreError: 365989:0:(tgt_mount.c:669:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [10418.699231] LustreError: 342768:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [10418.713033] LustreError: 342768:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [10418.760674] LustreError: 365994:0:(lod_lov.c:207:lod_add_device()) lustre-MDT0001-osp-MDT0000: cannot connect to next dev lustre-MDT0001_UUID (-114) [10418.769222] LustreError: 365994:0:(obd_config.c:1997:class_config_llog_handler()) MGC192.168.202.124@tcp: cfg command failed: rc = -114 [10418.774118] Lustre: cmd=cf014 0:lustre-MDT0000-mdtlov 1:lustre-MDT0001_UUID 2:1 3:1 [10418.774118] [10731.490100] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [10731.496984] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10731.511189] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10737.430890] LustreError: 375292:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup [10737.441773] LustreError: 375292:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [10737.523606] Lustre: server umount lustre-MDT0000 complete [10740.195061] LustreError: 342770:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10740.219017] LustreError: 342770:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 5 previous similar messages [10740.928091] LustreError: 345636:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1764801651 with bad export cookie 1530300191847819005 [10740.929578] LustreError: MGC192.168.202.124@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10740.954461] LustreError: 345636:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 2 previous similar messages [10745.312728] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10745.332904] Lustre: Skipped 2 previous similar messages [10747.195709] LustreError: 375494:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [10747.207811] LustreError: 375494:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [10747.547196] Lustre: server umount lustre-MDT0001 complete [10752.104364] Lustre: server umount lustre-OST0000 complete [10762.593070] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing set_hostid [10770.998425] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing load_modules_local [10778.017752] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [10782.619497] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [10787.869598] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [10794.268065] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [10800.534259] Lustre: DEBUG MARKER: == conf-sanity test 120: cross-target rename should not create bad symlinks ========================================================== 17:41:50 (1764801710) [10806.671589] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [10806.738489] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10806.909906] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [10806.941410] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [10807.013146] Lustre: lustre-MDT0000: new disk, initializing [10807.091039] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10807.105629] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [10810.668239] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing set_default_debug -1 all [10819.465753] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [10819.517469] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [10819.578927] Lustre: 378234:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [10819.694946] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [10819.697223] Lustre: Skipped 1 previous similar message [10819.789872] Lustre: lustre-MDT0001: new disk, initializing [10819.942569] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10819.982865] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [10819.999237] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [10824.068560] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing set_default_debug -1 all [10831.177961] Lustre: DEBUG MARKER: oleg224-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [10834.346718] Lustre: DEBUG MARKER: oleg224-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [10840.213280] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [10840.279578] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [10840.517408] Lustre: lustre-OST0000: new disk, initializing [10840.520459] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [10840.607593] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10845.878272] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing set_default_debug -1 all [10846.322613] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [10846.339387] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [10846.480875] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [10852.817925] Lustre: DEBUG MARKER: oleg224-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [10855.297038] Lustre: 378239:0:(osd_internal.h:1360:osd_trans_exec_op()) lustre-MDT0001: opcode 2: before 507 < left 525, rollback = 2 [10855.316707] Lustre: 378239:0:(osd_handler.c:2081:osd_trans_dump_creds()) create: 3/12/6, destroy: 0/0/0 [10855.328254] Lustre: 378239:0:(osd_handler.c:2088:osd_trans_dump_creds()) attr_set: 3/3/0, xattr_set: 6/525/0 [10855.346473] Lustre: 378239:0:(osd_handler.c:2098:osd_trans_dump_creds()) write: 7/153/0, punch: 0/0/0, quota 1/3/0 [10855.351277] Lustre: 378239:0:(osd_handler.c:2105:osd_trans_dump_creds()) insert: 7/135/3, delete: 0/0/0 [10855.355299] Lustre: 378239:0:(osd_handler.c:2112:osd_trans_dump_creds()) ref_add: 1/1/0, ref_del: 0/0/0 [10861.024701] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [10861.033196] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10861.050854] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10861.061122] Lustre: Skipped 4 previous similar messages [10864.417750] LustreError: 380674:0:(obd_class.h:479:obd_check_dev()) Device 22 not setup [10864.438551] LustreError: 380674:0:(obd_class.h:479:obd_check_dev()) Skipped 11 previous similar messages [10864.508700] Lustre: server umount lustre-MDT0000 complete [10867.173239] LustreError: 380588:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10867.190698] LustreError: 380588:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [10867.436162] LustreError: 378225:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1764801777 with bad export cookie 1530300191847820972 [10867.442643] LustreError: MGC192.168.202.124@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10867.447494] LustreError: 378225:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 3 previous similar messages [10867.756688] Lustre: server umount lustre-MDT0001 complete [10880.048748] Lustre: server umount lustre-OST0000 complete [10903.211714] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing set_hostid [10911.084221] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing load_modules_local [10919.523282] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [10925.098838] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [10930.659617] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [10936.811249] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [10938.436218] Lustre: DEBUG MARKER: == conf-sanity test complete, duration 10789 sec ========= 17:44:08 (1764801848) [10939.857747] Lustre: DEBUG MARKER: === conf-sanity: start cleanup 17:44:09 (1764801849) === [10942.546666] Lustre: DEBUG MARKER: === conf-sanity: finish cleanup 17:44:12 (1764801852) === [10975.709656] Lustre: DEBUG MARKER: oleg224-server.virtnet: executing unload_modules_local [10978.265018] Key type lgssc unregistered [10978.489486] LNet: 386197:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [10978.505794] LNetError: 386197:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [10978.525379] LNet: Removed LNI 192.168.202.124@tcp [10979.269492] Key type .llcrypt unregistered [10979.271781] Key type ._llcrypt unregistered