[ 0.000000] Linux version 4.18.0rh8.10-debug (green@maintenance) (gcc version 8.5.0 20210514 (Red Hat 8.5.0-26) (GCC)) #2 SMP Mon Jul 14 01:24:22 EDT 2025 [ 0.000000] Command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers' [ 0.000000] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256 [ 0.000000] x86/fpu: Enabled xstate features 0x7, context size is 832 bytes, using 'standard' format. [ 0.000000] signal: max sigframe size: 1776 [ 0.000000] BIOS-provided physical RAM map: [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x00000000bffcdfff] usable [ 0.000000] BIOS-e820: [mem 0x00000000bffce000-0x00000000bfffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x0000000146dfffff] usable [ 0.000000] NX (Execute Disable) protection: active [ 0.000000] SMBIOS 2.8 present. [ 0.000000] DMI: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.16.2-1.fc38 04/01/2014 [ 0.000000] Hypervisor detected: KVM [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00 [ 0.000000] kvm-clock: using sched offset of 496621295 cycles [ 0.000000] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns [ 0.000000] tsc: Detected 2399.998 MHz processor [ 0.000000] last_pfn = 0x146e00 max_arch_pfn = 0x400000000 [ 0.000000] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT [ 0.000000] last_pfn = 0xbffce max_arch_pfn = 0x400000000 [ 0.000000] found SMP MP-table at [mem 0x000f5b30-0x000f5b3f] [ 0.000000] RAMDISK: [mem 0xbcc54000-0xbffbffff] [ 0.000000] ACPI: Early table checksum verification disabled [ 0.000000] ACPI: RSDP 0x00000000000F5950 000014 (v00 BOCHS ) [ 0.000000] ACPI: RSDT 0x00000000BFFE1BB7 000034 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACP 0x00000000BFFE1A53 000074 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: DSDT 0x00000000BFFE0040 001A13 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACS 0x00000000BFFE0000 000040 [ 0.000000] ACPI: APIC 0x00000000BFFE1AC7 000090 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: HPET 0x00000000BFFE1B57 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: WAET 0x00000000BFFE1B8F 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: Reserving FACP table memory at [mem 0xbffe1a53-0xbffe1ac6] [ 0.000000] ACPI: Reserving DSDT table memory at [mem 0xbffe0040-0xbffe1a52] [ 0.000000] ACPI: Reserving FACS table memory at [mem 0xbffe0000-0xbffe003f] [ 0.000000] ACPI: Reserving APIC table memory at [mem 0xbffe1ac7-0xbffe1b56] [ 0.000000] ACPI: Reserving HPET table memory at [mem 0xbffe1b57-0xbffe1b8e] [ 0.000000] ACPI: Reserving WAET table memory at [mem 0xbffe1b8f-0xbffe1bb6] [ 0.000000] No NUMA configuration found [ 0.000000] Faking a node at [mem 0x0000000000000000-0x0000000146dfffff] [ 0.000000] NODE_DATA(0) allocated [mem 0x1465a3000-0x1465cdfff] [ 0.000000] Reserving 256MB of memory at 2752MB for crashkernel (System RAM: 4205MB) [ 0.000000] Zone ranges: [ 0.000000] DMA [mem 0x0000000000001000-0x0000000000ffffff] [ 0.000000] DMA32 [mem 0x0000000001000000-0x00000000ffffffff] [ 0.000000] Normal [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Device empty [ 0.000000] Movable zone start for each node [ 0.000000] Early memory node ranges [ 0.000000] node 0: [mem 0x0000000000001000-0x000000000009efff] [ 0.000000] node 0: [mem 0x0000000000100000-0x00000000bffcdfff] [ 0.000000] node 0: [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Zeroed struct page in unavailable ranges: 4756 pages [ 0.000000] Initmem setup node 0 [mem 0x0000000000001000-0x0000000146dfffff] [ 0.000000] ACPI: PM-Timer IO Port: 0x608 [ 0.000000] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1]) [ 0.000000] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23 [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level) [ 0.000000] Using ACPI (MADT) for SMP configuration information [ 0.000000] ACPI: HPET id: 0x8086a201 base: 0xfed00000 [ 0.000000] TSC deadline timer available [ 0.000000] smpboot: Allowing 4 CPUs, 0 hotplug CPUs [ 0.000000] kvm-guest: KVM setup pv remote TLB flush [ 0.000000] kvm-guest: setup PV sched yield [ 0.000000] PM: Registered nosave memory: [mem 0x00000000-0x00000fff] [ 0.000000] PM: Registered nosave memory: [mem 0x0009f000-0x0009ffff] [ 0.000000] PM: Registered nosave memory: [mem 0x000a0000-0x000effff] [ 0.000000] PM: Registered nosave memory: [mem 0x000f0000-0x000fffff] [ 0.000000] PM: Registered nosave memory: [mem 0xbffce000-0xbfffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xc0000000-0xfeffbfff] [ 0.000000] PM: Registered nosave memory: [mem 0xfeffc000-0xfeffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xff000000-0xfffbffff] [ 0.000000] PM: Registered nosave memory: [mem 0xfffc0000-0xffffffff] [ 0.000000] [mem 0xc0000000-0xfeffbfff] available for PCI devices [ 0.000000] Booting paravirtualized kernel on KVM [ 0.000000] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns [ 0.000000] setup_percpu: NR_CPUS:8192 nr_cpumask_bits:4 nr_cpu_ids:4 nr_node_ids:1 [ 0.000000] percpu: Embedded 63 pages/cpu s221184 r8192 d28672 u524288 [ 0.000000] kvm-guest: PV spinlocks enabled [ 0.000000] PV qspinlock hash table entries: 256 (order: 0, 4096 bytes, linear) [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 1059606 [ 0.000000] Policy zone: Normal [ 0.000000] Kernel command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] Specific versions of hardware are certified with Red Hat Enterprise Linux 8. Please see the list of hardware certified with Red Hat Enterprise Linux 8 at https://catalog.redhat.com. [ 0.000000] audit: disabled (until reboot) [ 0.000000] software IO TLB: area num 4. [ 0.000000] Memory: 2895240K/4306352K available (18435K kernel code, 11221K rwdata, 7248K rodata, 2908K init, 18040K bss, 524584K reserved, 0K cma-reserved) [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1 [ 0.000000] kmemleak: Kernel memory leak detector disabled [ 0.000000] ftrace: allocating 41240 entries in 162 pages [ 0.000000] ftrace: allocated 162 pages with 3 groups [ 0.000000] rcu: Hierarchical RCU implementation. [ 0.000000] rcu: RCU event tracing is enabled. [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=8192 to nr_cpu_ids=4. [ 0.000000] rcu: RCU callback double-/use-after-free debug enabled. [ 0.000000] Rude variant of Tasks RCU enabled. [ 0.000000] Tracing variant of Tasks RCU enabled. [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies. [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4 [ 0.000000] NR_IRQS: 524544, nr_irqs: 456, preallocated irqs: 16 [ 0.000000] random: get_random_bytes called from start_kernel+0x622/0x9a8 with crng_init=0 [ 0.001000] Console: colour *CGA 80x25 [ 0.001000] printk: console [ttyS1] enabled [ 0.001000] ACPI: Core revision 20220331 [ 0.001000] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns [ 0.001014] APIC: Switch to symmetric I/O mode setup [ 0.002392] x2apic enabled [ 0.003006] Switched APIC routing to physical x2apic. [ 0.004009] kvm-guest: setup PV IPIs [ 0.007662] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1 [ 0.008000] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 0.008016] Calibrating delay loop (skipped) preset value.. 4799.99 BogoMIPS (lpj=2399998) [ 0.009007] pid_max: default: 32768 minimum: 301 [ 0.010122] LSM: Security Framework initializing [ 0.011036] Yama: becoming mindful. [ 0.012068] SELinux: Initializing. [ 0.014062] *** VALIDATE selinux *** [ 0.022250] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, vmalloc) [ 0.027609] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, vmalloc) [ 0.028189] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.029093] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.030147] *** VALIDATE tmpfs *** [ 0.032310] *** VALIDATE proc *** [ 0.033263] *** VALIDATE cgroup *** [ 0.034006] *** VALIDATE cgroup2 *** [ 0.035283] x86/cpu: User Mode Instruction Prevention (UMIP) activated [ 0.036161] Last level iTLB entries: 4KB 0, 2MB 0, 4MB 0 [ 0.037004] Last level dTLB entries: 4KB 0, 2MB 0, 4MB 0, 1GB 0 [ 0.038031] Spectre V2 : User space: Vulnerable [ 0.039006] Speculative Store Bypass: Vulnerable [ 0.043691] debug: unmapping init [mem 0xffffffffa0a59000-0xffffffffa0a60fff] [ 0.046147] smpboot: CPU0: Intel(R) Xeon(R) CPU E5-2695 v2 @ 2.40GHz (family: 0x6, model: 0x3e, stepping: 0x4) [ 0.047866] Performance Events: IvyBridge events, full-width counters, Intel PMU driver. [ 0.048026] ... version: 2 [ 0.049014] ... bit width: 48 [ 0.050008] ... generic registers: 4 [ 0.051009] ... value mask: 0000ffffffffffff [ 0.052013] ... max period: 00007fffffffffff [ 0.053012] ... fixed-purpose events: 3 [ 0.054009] ... event mask: 000000070000000f [ 0.056346] rcu: Hierarchical SRCU implementation. [ 0.058992] smp: Bringing up secondary CPUs ... [ 0.060664] x86: Booting SMP configuration: [ 0.061018] .... node #0, CPUs: #1 #2 #3 [ 0.064554] smp: Brought up 1 node, 4 CPUs [ 0.066010] smpboot: Max logical packages: 1 [ 0.067018] smpboot: Total of 4 processors activated (19199.98 BogoMIPS) [ 0.219023] node 0 deferred pages initialised in 150ms [ 0.222009] devtmpfs: initialized [ 0.223211] x86/mm: Memory block size: 128MB [ 0.225579] gcov: version magic: 0x41383552 [ 0.227213] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns [ 0.228068] futex hash table entries: 1024 (order: 4, 65536 bytes, vmalloc) [ 0.229304] pinctrl core: initialized pinctrl subsystem [ 0.230127] [ 0.230674] ************************************************************* [ 0.233008] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.235007] ** ** [ 0.237008] ** IOMMU DebugFS SUPPORT HAS BEEN ENABLED IN THIS KERNEL ** [ 0.239008] ** ** [ 0.241008] ** This means that this kernel is built to expose internal ** [ 0.243007] ** IOMMU data structures, which may compromise security on ** [ 0.246013] ** your system. ** [ 0.247006] ** ** [ 0.249007] ** If you see this message and you are not debugging the ** [ 0.251008] ** kernel, report this immediately to your vendor! ** [ 0.253008] ** ** [ 0.255006] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.257011] ************************************************************* [ 0.259863] NET: Registered protocol family 16 [ 0.261451] DMA: preallocated 512 KiB GFP_KERNEL pool for atomic allocations [ 0.263026] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations [ 0.265172] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations [ 0.268074] cpuidle: using governor menu [ 0.270089] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5 [ 0.271410] PCI: Using configuration type 1 for base access [ 0.273111] core: PMU erratum BJ122, BV98, HSD29 worked around, HT is on [ 0.281143] HugeTLB registered 1.00 GiB page size, pre-allocated 0 pages [ 0.283018] HugeTLB registered 2.00 MiB page size, pre-allocated 0 pages [ 0.286075] cryptd: max_cpu_qlen set to 1000 [ 0.288057] ACPI: Added _OSI(Module Device) [ 0.289022] ACPI: Added _OSI(Processor Device) [ 0.289843] ACPI: Added _OSI(3.0 _SCP Extensions) [ 0.290053] ACPI: Added _OSI(Processor Aggregator Device) [ 0.293279] ACPI: 1 ACPI AML tables successfully acquired and loaded [ 0.298524] ACPI: Interpreter enabled [ 0.299047] ACPI: PM: (supports S0 S3 S4 S5) [ 0.301009] ACPI: Using IOAPIC for interrupt routing [ 0.302093] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug [ 0.305376] ACPI: Enabled 2 GPEs in block 00 to 0F [ 0.315536] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff]) [ 0.317022] acpi PNP0A03:00: _OSC: OS supports [ASPM ClockPM Segments MSI HPX-Type3] [ 0.319013] acpi PNP0A03:00: _OSC: not requesting OS control; OS requires [ExtendedConfig ASPM ClockPM MSI] [ 0.322067] acpi PNP0A03:00: fail to add MMCONFIG information, can't access extended PCI configuration space under this bridge. [ 0.326264] acpiphp: Slot [2] registered [ 0.328093] acpiphp: Slot [3] registered [ 0.329076] acpiphp: Slot [4] registered [ 0.330068] acpiphp: Slot [5] registered [ 0.331105] acpiphp: Slot [6] registered [ 0.333083] acpiphp: Slot [7] registered [ 0.334118] acpiphp: Slot [8] registered [ 0.335110] acpiphp: Slot [9] registered [ 0.336098] acpiphp: Slot [10] registered [ 0.338107] acpiphp: Slot [11] registered [ 0.339075] acpiphp: Slot [12] registered [ 0.340063] acpiphp: Slot [13] registered [ 0.341080] acpiphp: Slot [14] registered [ 0.343087] acpiphp: Slot [15] registered [ 0.344074] acpiphp: Slot [16] registered [ 0.345080] acpiphp: Slot [17] registered [ 0.347083] acpiphp: Slot [18] registered [ 0.348080] acpiphp: Slot [19] registered [ 0.349081] acpiphp: Slot [20] registered [ 0.350073] acpiphp: Slot [21] registered [ 0.352064] acpiphp: Slot [22] registered [ 0.353090] acpiphp: Slot [23] registered [ 0.354063] acpiphp: Slot [24] registered [ 0.355069] acpiphp: Slot [25] registered [ 0.357075] acpiphp: Slot [26] registered [ 0.358122] acpiphp: Slot [27] registered [ 0.359078] acpiphp: Slot [28] registered [ 0.361086] acpiphp: Slot [29] registered [ 0.362063] acpiphp: Slot [30] registered [ 0.364063] acpiphp: Slot [31] registered [ 0.365062] PCI host bridge to bus 0000:00 [ 0.366011] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window] [ 0.368011] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window] [ 0.370009] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window] [ 0.371011] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window] [ 0.372009] pci_bus 0000:00: root bus resource [mem 0x180000000-0x1ffffffff window] [ 0.374010] pci_bus 0000:00: root bus resource [bus 00-ff] [ 0.375127] pci 0000:00:00.0: [8086:1237] type 00 class 0x060000 [ 0.377658] pci 0000:00:01.0: [8086:7000] type 00 class 0x060100 [ 0.380826] pci 0000:00:01.1: [8086:7010] type 00 class 0x010180 [ 0.387941] pci 0000:00:01.1: reg 0x20: [io 0xc320-0xc32f] [ 0.393059] pci 0000:00:01.1: legacy IDE quirk: reg 0x10: [io 0x01f0-0x01f7] [ 0.396037] pci 0000:00:01.1: legacy IDE quirk: reg 0x14: [io 0x03f6] [ 0.398019] pci 0000:00:01.1: legacy IDE quirk: reg 0x18: [io 0x0170-0x0177] [ 0.400015] pci 0000:00:01.1: legacy IDE quirk: reg 0x1c: [io 0x0376] [ 0.402469] pci 0000:00:01.3: [8086:7113] type 00 class 0x068000 [ 0.403649] pci 0000:00:01.3: quirk: [io 0x0600-0x063f] claimed by PIIX4 ACPI [ 0.406028] pci 0000:00:01.3: quirk: [io 0x0700-0x070f] claimed by PIIX4 SMB [ 0.408603] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 [ 0.411825] pci 0000:00:02.0: reg 0x10: [io 0xc300-0xc31f] [ 0.420030] pci 0000:00:02.0: reg 0x20: [mem 0xfebe4000-0xfebe7fff 64bit pref] [ 0.425018] pci 0000:00:02.0: reg 0x30: [mem 0xfeb80000-0xfebbffff pref] [ 0.430979] pci 0000:00:05.0: [1af4:1001] type 00 class 0x010000 [ 0.440027] pci 0000:00:05.0: reg 0x10: [io 0xc000-0xc07f] [ 0.451026] pci 0000:00:05.0: reg 0x14: [mem 0xfebc0000-0xfebc0fff] [ 0.473037] pci 0000:00:05.0: reg 0x20: [mem 0xfebe8000-0xfebebfff 64bit pref] [ 0.490350] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 [ 0.498025] pci 0000:00:06.0: reg 0x10: [io 0xc080-0xc0ff] [ 0.505000] pci 0000:00:06.0: reg 0x14: [mem 0xfebc1000-0xfebc1fff] [ 0.517018] pci 0000:00:06.0: reg 0x20: [mem 0xfebec000-0xfebeffff 64bit pref] [ 0.527515] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 [ 0.536028] pci 0000:00:07.0: reg 0x10: [io 0xc100-0xc17f] [ 0.549022] pci 0000:00:07.0: reg 0x14: [mem 0xfebc2000-0xfebc2fff] [ 0.574034] pci 0000:00:07.0: reg 0x20: [mem 0xfebf0000-0xfebf3fff 64bit pref] [ 0.591955] pci 0000:00:08.0: [1af4:1001] type 00 class 0x010000 [ 0.598019] pci 0000:00:08.0: reg 0x10: [io 0xc180-0xc1ff] [ 0.605013] pci 0000:00:08.0: reg 0x14: [mem 0xfebc3000-0xfebc3fff] [ 0.618014] pci 0000:00:08.0: reg 0x20: [mem 0xfebf4000-0xfebf7fff 64bit pref] [ 0.628262] pci 0000:00:09.0: [1af4:1001] type 00 class 0x010000 [ 0.634015] pci 0000:00:09.0: reg 0x10: [io 0xc200-0xc27f] [ 0.642047] pci 0000:00:09.0: reg 0x14: [mem 0xfebc4000-0xfebc4fff] [ 0.658017] pci 0000:00:09.0: reg 0x20: [mem 0xfebf8000-0xfebfbfff 64bit pref] [ 0.668921] pci 0000:00:0a.0: [1af4:1001] type 00 class 0x010000 [ 0.675020] pci 0000:00:0a.0: reg 0x10: [io 0xc280-0xc2ff] [ 0.682025] pci 0000:00:0a.0: reg 0x14: [mem 0xfebc5000-0xfebc5fff] [ 0.695021] pci 0000:00:0a.0: reg 0x20: [mem 0xfebfc000-0xfebfffff 64bit pref] [ 0.706727] ACPI: PCI: Interrupt link LNKA configured for IRQ 10 [ 0.710484] ACPI: PCI: Interrupt link LNKB configured for IRQ 10 [ 0.713367] ACPI: PCI: Interrupt link LNKC configured for IRQ 11 [ 0.716462] ACPI: PCI: Interrupt link LNKD configured for IRQ 11 [ 0.719216] ACPI: PCI: Interrupt link LNKS configured for IRQ 9 [ 0.725218] iommu: Default domain type: Passthrough [ 0.727440] SCSI subsystem initialized [ 0.728099] ACPI: bus type USB registered [ 0.729102] usbcore: registered new interface driver usbfs [ 0.731076] usbcore: registered new interface driver hub [ 0.732063] usbcore: registered new device driver usb [ 0.733153] pps_core: LinuxPPS API ver. 1 registered [ 0.735008] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti [ 0.737054] PTP clock support registered [ 0.740077] EDAC MC: Ver: 3.0.0 [ 0.742329] PCI: Using ACPI for IRQ routing [ 0.745338] NetLabel: Initializing [ 0.746012] NetLabel: domain hash size = 128 [ 0.747011] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO [ 0.749088] NetLabel: unlabeled traffic allowed by default [ 0.752168] vgaarb: loaded [ 0.753495] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0 [ 0.755016] hpet0: 3 comparators, 64-bit 100.000000 MHz counter [ 0.760621] clocksource: Switched to clocksource kvm-clock [ 0.867695] VFS: Disk quotas dquot_6.6.0 [ 0.869308] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) [ 0.871636] *** VALIDATE ramfs *** [ 0.872787] *** VALIDATE hugetlbfs *** [ 0.874241] pnp: PnP ACPI init [ 0.876463] pnp: PnP ACPI: found 6 devices [ 0.893169] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns [ 0.896172] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window] [ 0.898153] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window] [ 0.899967] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window] [ 0.902154] pci_bus 0000:00: resource 7 [mem 0xc0000000-0xfebfffff window] [ 0.904336] pci_bus 0000:00: resource 8 [mem 0x180000000-0x1ffffffff window] [ 0.906760] NET: Registered protocol family 2 [ 0.909129] IP idents hash table entries: 131072 (order: 8, 1048576 bytes, vmalloc) [ 0.913670] tcp_listen_portaddr_hash hash table entries: 4096 (order: 5, 163840 bytes, vmalloc) [ 0.916904] TCP established hash table entries: 65536 (order: 7, 524288 bytes, vmalloc) [ 0.921713] TCP bind hash table entries: 65536 (order: 9, 2097152 bytes, vmalloc) [ 0.924863] TCP: Hash tables configured (established 65536 bind 65536) [ 0.927586] MPTCP token hash table entries: 8192 (order: 6, 393216 bytes, vmalloc) [ 0.930396] UDP hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 0.932979] UDP-Lite hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 0.935847] NET: Registered protocol family 1 [ 0.939146] RPC: Registered named UNIX socket transport module. [ 0.941068] RPC: Registered udp transport module. [ 0.942676] RPC: Registered tcp transport module. [ 0.944193] RPC: Registered tcp NFSv4.1 backchannel transport module. [ 0.946314] NET: Registered protocol family 44 [ 0.947823] pci 0000:00:00.0: Limiting direct PCI/PCI transfers [ 0.949675] pci 0000:00:01.0: PIIX3: Enabling Passive Release [ 0.951509] pci 0000:00:01.0: Activating ISA DMA hang workarounds [ 0.953570] PCI: CLS 0 bytes, default 64 [ 0.955047] Unpacking initramfs... [ 2.844405] debug: unmapping init [mem 0xffff9405bcc54000-0xffff9405bffbffff] [ 2.851302] PCI-DMA: Using software bounce buffering for IO (SWIOTLB) [ 2.855049] software IO TLB: mapped [mem 0x00000000a8000000-0x00000000ac000000] (64MB) [ 2.859385] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 3.388290] Initialise system trusted keyrings [ 3.389763] Key type blacklist registered [ 3.392069] workingset: timestamp_bits=36 max_order=20 bucket_order=0 [ 3.400789] zbud: loaded [ 3.404016] *** VALIDATE nfs *** [ 3.405082] *** VALIDATE nfs4 *** [ 3.406359] pstore: using deflate compression [ 3.409948] Platform Keyring initialized [ 3.610872] NET: Registered protocol family 38 [ 3.617619] Key type asymmetric registered [ 3.622523] Asymmetric key parser 'x509' registered [ 3.631620] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 247) [ 3.638284] io scheduler mq-deadline registered [ 3.641199] io scheduler kyber registered [ 3.645231] io scheduler bfq registered [ 3.650534] atomic64_test: passed for x86-64 platform with CX8 and with SSE [ 3.653160] shpchp: Standard Hot Plug PCI Controller Driver version: 0.4 [ 3.656410] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0 [ 3.659289] ACPI: Power Button [PWRF] [ 3.813537] ACPI: \_SB_.LNKB: Enabled at IRQ 10 [ 3.913381] ACPI: \_SB_.LNKA: Enabled at IRQ 11 [ 4.252372] ACPI: \_SB_.LNKC: Enabled at IRQ 11 [ 4.386626] ACPI: \_SB_.LNKD: Enabled at IRQ 10 [ 4.686646] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled [ 4.728108] 00:03: ttyS1 at I/O 0x2f8 (irq = 3, base_baud = 115200) is a 16550A [ 4.762770] 00:04: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A [ 4.771286] Non-volatile memory driver v1.3 [ 4.772930] Linux agpgart interface v0.103 [ 4.812879] virtio_blk virtio1: [vda] 133544 512-byte logical blocks (68.4 MB/65.2 MiB) [ 4.817736] vda: detected capacity change from 0 to 68374528 [ 4.860294] virtio_blk virtio2: [vdb] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) [ 4.862891] vdb: detected capacity change from 0 to 1073741824 [ 4.891673] virtio_blk virtio3: [vdc] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 4.898854] vdc: detected capacity change from 0 to 2621440000 [ 4.923595] virtio_blk virtio4: [vdd] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 4.926205] vdd: detected capacity change from 0 to 2621440000 [ 4.945531] virtio_blk virtio5: [vde] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 4.951773] vde: detected capacity change from 0 to 4294967296 [ 4.977423] virtio_blk virtio6: [vdf] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 4.983636] vdf: detected capacity change from 0 to 4294967296 [ 5.000780] libphy: Fixed MDIO Bus: probed [ 5.012120] usbcore: registered new interface driver usbserial_generic [ 5.014207] usbserial: USB Serial support registered for generic [ 5.018251] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12 [ 5.023312] serio: i8042 KBD port at 0x60,0x64 irq 1 [ 5.025269] serio: i8042 AUX port at 0x60,0x64 irq 12 [ 5.027787] mousedev: PS/2 mouse device common for all mice [ 5.036180] rtc_cmos 00:05: RTC can wake from S4 [ 5.039683] rtc_cmos 00:05: registered as rtc0 [ 5.040332] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input1 [ 5.050365] rtc_cmos 00:05: alarms up to one day, y3k, 242 bytes nvram, hpet irqs [ 5.050416] intel_pstate: CPU model not supported [ 5.057950] hid: raw HID events driver (C) Jiri Kosina [ 5.069778] hpet1: lost 1 rtc interrupts [ 5.097936] usbcore: registered new interface driver usbhid [ 5.100723] usbhid: USB HID core driver [ 5.102083] drop_monitor: Initializing network drop monitor service [ 5.106642] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input4 [ 5.111189] Initializing XFRM netlink socket [ 5.115857] NET: Registered protocol family 10 [ 5.121279] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input3 [ 5.126596] Segment Routing with IPv6 [ 5.127848] NET: Registered protocol family 17 [ 5.131114] mpls_gso: MPLS GSO support [ 5.137338] RAS: Correctable Errors collector initialized. [ 5.139468] AVX version of gcm_enc/dec engaged. [ 5.141082] AES CTR mode by8 optimization enabled [ 5.264731] sched_clock: Marking stable (5264580949, 0)->(6181363164, -916782215) [ 5.277652] registered taskstats version 1 [ 5.289780] Loading compiled-in X.509 certificates [ 5.291680] zswap: loaded using pool lzo/zbud [ 5.345196] Key type big_key registered [ 5.368344] Key type encrypted registered [ 5.369740] ima: No TPM chip found, activating TPM-bypass! [ 5.376947] ima: Allocated hash algorithm: sha1 [ 5.378368] ima: No architecture policies found [ 5.379678] evm: Initialising EVM extended attributes: [ 5.381211] evm: security.selinux [ 5.382180] evm: security.ima [ 5.382982] evm: security.capability [ 5.383955] evm: HMAC attrs: 0x1 [ 5.386101] rtc_cmos 00:05: setting system clock to 2025-08-12 22:56:10 UTC (1755039370) [ 5.394212] debug: unmapping init [mem 0xffffffffa1a03000-0xffffffffa1bfffff] [ 5.396647] debug: unmapping init [mem 0xffffffffa0782000-0xffffffffa0a58fff] [ 5.407082] Write protecting the kernel read-only data: 28672k [ 5.412760] debug: unmapping init [mem 0xffffffff9ee03000-0xffffffff9effffff] [ 5.415355] debug: unmapping init [mem 0xffffffff9f714000-0xffffffff9f7fffff] [ 5.452848] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 5.460988] systemd[1]: Detected virtualization kvm. [ 5.462627] systemd[1]: Detected architecture x86-64. [ 5.464823] systemd[1]: Running in initial RAM disk. Welcome to Rocky Linux 8.10 (Green Obsidian) dracut-049-233.git20240115.el8 (Initramfs)! [ 5.495572] systemd[1]: No hostname configured. [ 5.498068] systemd[1]: Set hostname to . [ 5.499865] random: systemd: uninitialized urandom read (16 bytes read) [ 5.504838] systemd[1]: Initializing machine ID from random generator. [ 5.651655] random: ln: uninitialized urandom read (6 bytes read) [ 5.815871] random: systemd: uninitialized urandom read (16 bytes read) [ 5.818332] systemd[1]: Listening on Journal Socket. [ OK ] Listening on Journal Socket. [ 5.832933] systemd[1]: Started Memstrack Anylazing Service. [ OK ] Started Memstrack Anylazing Service. [ 5.856735] systemd[1]: Reached target Initrd Root Device. [ OK ] Reached target Initrd Root Device. [ OK ] Reached target Slices. [ OK ] Listening on udev Control Socket. [ OK ] Reached target Swap. [ OK ] Listening on Journal Socket (/dev/log). Starting Journal Service... [ OK ] Reached target Timers. Starting Create list of required st…ce nodes for the current kernel... [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Reached target Local Encrypted Volumes. [ OK ] Reached target Paths. Starting Setup Virtual Console... [ OK ] Listening on udev Kernel Socket. [ OK ] Reached target Sockets. Starting Apply Kernel Variables... [ OK ] Reached target Local File Systems. Starting Create Volatile Files and Directories... [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Started Setup Virtual Console. [ OK ] Started Apply Kernel Variables. [ OK ] Started Create Volatile Files and Directories. Starting dracut cmdline hook... Starting Create Static Device Nodes in /dev... [ OK ] Started Create Static Device Nodes in /dev. [ OK ] Started Journal Service. [ OK ] Started dracut cmdline hook. Starting dracut pre-udev hook... [ 7.430722] device-mapper: uevent: version 1.0.3 [ 7.434345] device-mapper: ioctl: 4.46.0-ioctl (2022-02-22) initialised: dm-devel@redhat.com [ OK ] Started dracut pre-udev hook. Starting udev Kernel Device Manager... [ OK ] Started udev Kernel Device Manager. Starting dracut pre-trigger hook... [ OK ] Started dracut pre-trigger hook. Starting udev Coldplug all Devices... Mounting Kernel Configuration File System... [ OK ] Mounted Kernel Configuration File System. [ OK ] Started udev Coldplug all Devices. Starting dracut initqueue hook... [ OK ] Reached target System Initialization. [ OK ] Reached target Basic System. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. [ 9.798211] virtio_net virtio0 ens2: renamed from eth0 [ 9.893970] random: fast init done [ 10.324543] scsi host0: ata_piix [ 10.420393] scsi host1: ata_piix [ 10.425689] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc320 irq 14 [ 10.432189] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc328 irq 15 [ 15.899480] random: crng init done [ 15.900592] random: 7 urandom warning(s) missed due to ratelimiting [ 17.437175] dracut-initqueue[599]: RTNETLINK answers: File exists Starting nbd nbd0... [ OK ] Started nbd nbd0. [ OK ] Started dracut initqueue hook. Mounting /sysroot... [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. [ 18.671971] EXT4-fs (nbd0): mounted filesystem with ordered data mode. Opts: (null) [ OK ] Mounted /sysroot. [ OK ] Reached target Initrd Root File System. Starting Reload Configuration from the Real Root... [ OK ] Started Reload Configuration from the Real Root. [ OK ] Reached target Initrd File Systems. [ OK ] Reached target Initrd Default Target. Starting dracut pre-pivot and cleanup hook... [ OK ] Started dracut pre-pivot and cleanup hook. Starting Cleaning Up and Shutting Down Daemons... [ OK ] Stopped target Timers. [ OK ] Stopped dracut pre-pivot and cleanup hook. [ OK ] Stopped target Remote File Systems. [ OK ] Stopped target Remote File Systems (Pre). [ OK ] Stopped target Initrd Default Target. [ OK ] Stopped target Initrd Root Device. [ OK ] Stopped dracut initqueue hook. Stopping Hardware RNG Entropy Gatherer Daemon... [ OK ] Stopped Hardware RNG Entropy Gatherer Daemon. [ OK ] Stopped target Basic System. [ OK ] Stopped target Sockets. [ OK ] Stopped target Paths. [ OK ] Stopped target Slices. [ OK ] Stopped target System Initialization. [ OK ] Stopped udev Coldplug all Devices. [ OK ] Stopped dracut pre-trigger hook. Stopping udev Kernel Device Manager... [ OK ] Stopped target Swap. [ OK ] Stopped Apply Kernel Variables. [ OK ] Stopped target Local Encrypted Volumes. [ OK ] Stopped Dispatch Password Requests to Console Directory Watch. [ OK ] Stopped Create Volatile Files and Directories. [ OK ] Stopped target Local File Systems. [ OK ] Stopped udev Kernel Device Manager. [ OK ] Started Cleaning Up and Shutting Down Daemons. [ OK ] Stopped dracut pre-udev hook. [ OK ] Stopped dracut cmdline hook. [ OK ] Stopped Create Static Device Nodes in /dev. [ OK ] Stopped Create list of required sta…vice nodes for the current kernel. [ OK ] Closed udev Kernel Socket. [ OK ] Closed udev Control Socket. Starting Cleanup udevd DB... [ OK ] Started Cleanup udevd DB. [ OK ] Reached target Switch Root. Starting Switch Root... [ 21.060925] printk: systemd: 26 output lines suppressed due to ratelimiting [ 21.761577] SELinux: Disabled at runtime. [ 21.868689] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 21.876317] systemd[1]: Detected virtualization kvm. [ 21.877897] systemd[1]: Detected architecture x86-64. Welcome to Rocky Linux 8.10 (Green Obsidian)! [ 22.974699] systemd[1]: initrd-switch-root.service: Succeeded. [ 22.982904] systemd[1]: Stopped Switch Root. [ OK ] Stopped Switch Root. [ 22.991670] systemd[1]: systemd-journald.service: Service has no hold-off time (RestartSec=0), scheduling restart. [ 23.000889] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. [ 23.007211] systemd[1]: Stopped Journal Service. [ OK ] Stopped Journal Service. [ 23.025873] systemd[1]: Starting Journal Service... Starting Journal Service... [ 23.035448] systemd[1]: Reached target rpc_pipefs.target. [ OK ] Reached target rpc_pipefs.target. Mounting POSIX Message Queue File System... Mounting Kernel Debug File System... [FAILED] Failed to set up automount Arbitrar…rmats File System Automount Point. See 'systemctl status proc-sys-fs-binfmt_misc.automount' for details. [ OK ] Listening on initctl Compatibility Named Pipe. [ OK ] Started Dispatch Password Requests to Console Directory Watch. Activating swap /dev/disk/by-label/SWAP... [ OK ] Stopped target Switch Root. [ OK ] Stopped target Initrd Root File System. [ OK ] Created slice User and Session Slice. Mounting Huge Pages File System... [ 23.226556] Adding 1048572k swap on /dev/vdb. Priority:-2 extents:1 across:1048572k FS Starting Apply Kernel Variables... [ OK ] Created slice system-serial\x2dgetty.slice. [ OK ] Created slice system-sshd\x2dkeygen.slice. [ OK ] Reached target Slices. [ OK ] Started Forward Password Requests to Wall Directory Watch. [ OK ] Stopped target Initrd File Systems. [ OK ] Listening on RPCbind Server Activation Socket. Starting Create list of required st…ce nodes for the current kernel... [ OK ] Listening on udev Kernel Socket. [ OK ] Reached target RPC Port Mapper. [ OK ] Reached target Local Encrypted Volumes. Starting Remount Root and Kernel File Systems... [ OK ] Listening on udev Control Socket. Starting udev Coldplug all Devices... [ OK ] Reached target Paths. [ OK ] Created slice system-getty.slice. [ OK ] Listening on Process Core Dump Socket. [ OK ] Started Journal Service. [ OK ] Mounted POSIX Message Queue File System. [ OK ] Mounted Kernel Debug File System. [ OK ] Activated swap /dev/disk/by-label/SWAP. [ OK ] Mounted Huge Pages File System. [ OK ] Started Apply Kernel Variables. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [FAILED] Failed to start Remount Root and Kernel File Systems. See 'systemctl status systemd-remount-fs.service' for details. Starting Configure read-only root support... Starting Create Static Device Nodes in /dev... [ OK ] Reached target Swap. Starting Flush Journal to Persistent Storage... [ OK ] Started Flush Journal to Persistent Storage. [ OK ] Started Create Static Device Nodes in /dev. Starting udev Kernel Device Manager... [ OK ] Reached target Local File Systems (Pre). Mounting /mnt... Mounting /home/green/git/lustre-release... [ OK ] Mounted /mnt. [ OK ] Started udev Coldplug all Devices. [ 24.236990] squashfs: version 4.0 (2009/01/31) Phillip Lougher [ OK ] Mounted /home/green/git/lustre-release. [ OK ] Started udev Kernel Device Manager. [ 24.860718] piix4_smbus 0000:00:01.3: SMBus Host Controller at 0x700, revision 0 [ 25.008364] input: PC Speaker as /devices/platform/pcspkr/input/input5 [ 25.525703] RAPL PMU: API unit is 2^-32 Joules, 0 fixed counters, 10737418240 ms ovfl timer [ 25.588067] EDAC sbridge: Ver: 1.1.2 [ 29.484727] Key type dns_resolver registered [* ] A start job is running for Configur…-only root support (7s / no limit)[ 30.138306] NFS: Registering the id_resolver key type [ 30.140083] Key type id_resolver registered [ 30.141502] Key type id_legacy registered [** ] A start job is running for Configur…-only root support (7s / no limit) [ OK ] Started Configure read-only root support. Starting Load/Save Random Seed... [ OK ] Reached target Local File Systems. Starting Mark the need to relabel after reboot... Starting Rebuild Dynamic Linker Cache... Starting Create Volatile Files and Directories... [ OK ] Started Load/Save Random Seed. [ OK ] Started Mark the need to relabel after reboot. [ OK ] Started Create Volatile Files and Directories. Starting RPC Bind... Starting Update UTMP about System Boot/Shutdown... [ OK ] Started Update UTMP about System Boot/Shutdown. [ OK ] Started RPC Bind. [ OK ] Started Rebuild Dynamic Linker Cache. Starting Update is Completed... [ OK ] Started Update is Completed. [ OK ] Reached target System Initialization. [ OK ] Listening on D-Bus System Message Bus Socket. [ OK ] Reached target Sockets. [ OK ] Started daily update of the root trust anchor for DNSSEC. [ OK ] Reached target Basic System. Starting Restore /run/initramfs on shutdown... [ OK ] Started irqbalance daemon. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. [ OK ] Started D-Bus System Message Bus. Starting Network Manager... [ OK ] Reached target sshd-keygen.target. Starting Login Service... [ OK ] Started dnf makecache --timer. [ OK ] Started Daily Cleanup of Temporary Directories. [ OK ] Reached target Timers. [ OK ] Started Restore /run/initramfs on shutdown. [ OK ] Started Network Manager. Starting Network Manager Wait Online... [ OK ] Reached target Network. Starting Dynamic System Tuning Daemon... Starting OpenSSH server daemon... Starting GSSAPI Proxy Daemon... Starting Hostname Service... [ OK ] Started Login Service. [ OK ] Started GSSAPI Proxy Daemon. [ OK ] Started OpenSSH server daemon. [ OK ] Reached target NFS client services. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Starting Permit User Sessions... [ OK ] Started Permit User Sessions. [ OK ] Started Command Scheduler. [ OK ] Started Serial Getty on ttyS1. [ OK ] Started Getty on tty1. [ OK ] Started Serial Getty on ttyS0. [ OK ] Reached target Login Prompts. [ OK ] Started Hostname Service. Starting Network Manager Script Dispatcher Service... [ OK ] Started Network Manager Script Dispatcher Service. [ OK ] Started Network Manager Wait Online. [ OK ] Reached target Network is Online. Starting Crash recovery kernel arming... Starting System Logging Service... Starting Notify NFS peers of a restart... [ OK ] Started Notify NFS peers of a restart. [ OK ] Started System Logging Service. Starting Authorization Manager... [ OK ] Started Dynamic System Tuning Daemon. Rocky Linux 8.10 (Green Obsidian) Kernel 4.18.0rh8.10-debug on an x86_64 oleg208-server login: [ 57.812249] libcfs: loading out-of-tree module taints kernel. [ 57.820777] Key type ._llcrypt registered [ 57.822521] Key type .llcrypt registered [ 57.864457] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing set_hostid [ 64.753472] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing load_modules_local [ 65.287583] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 65.294594] alg: No test for adler32 (adler32-zlib) [ 66.278581] Lustre: Lustre: Build Version: 2.16.56_30_ga643f08 [ 66.557214] LNet: Added LNI 192.168.202.108@tcp [8/256/0/180] [ 66.558554] LNet: Accept secure, port 988 [ 68.175131] Key type lgssc registered [ 68.626742] Lustre: Echo OBD driver; http://www.lustre.org/ [ 74.557048] ZFS: Loaded module v2.3.2-1, ZFS pool version 5000, ZFS filesystem version 5 [ 75.515822] LDISKFS-fs (vdc): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 78.888806] LDISKFS-fs (vdd): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 81.381972] LDISKFS-fs (vde): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 83.984258] LDISKFS-fs (vdf): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 89.534543] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing load_modules_local [ 93.962883] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 93.989542] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 94.002646] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 95.100688] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 95.113112] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 95.148816] Lustre: lustre-MDT0000: new disk, initializing [ 95.170820] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 95.178570] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 96.452516] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 101.553438] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 101.581958] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 101.610533] Lustre: 6454:0:(mgs_llog.c:1345:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 101.623353] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 101.626303] Lustre: Skipped 1 previous similar message [ 101.665281] Lustre: lustre-MDT0001: new disk, initializing [ 101.687089] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 101.694863] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 101.700022] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 103.003735] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 105.283450] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 108.274771] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 108.295348] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 108.369978] Lustre: lustre-OST0000: new disk, initializing [ 108.373864] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 108.388668] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 109.546253] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 109.549912] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 109.568941] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 110.171580] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 115.624762] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 115.656770] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 115.696543] Lustre: lustre-OST0001: new disk, initializing [ 115.698332] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 115.717586] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 116.047134] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 116.051830] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 116.078850] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 117.658967] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 123.577060] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 128.279211] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 135.119637] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing check_logdir /tmp/testlogs/ [ 136.605701] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing yml_node [ 138.430972] Lustre: DEBUG MARKER: Client: 2.16.56.30 [ 139.430551] Lustre: DEBUG MARKER: MDS: 2.16.56.30 [ 140.429875] Lustre: DEBUG MARKER: OSS: 2.16.56.30 [ 141.045898] Lustre: DEBUG MARKER: -----============= acceptance-small: conf-sanity ============----- Tue Aug 12 18:58:26 EDT 2025 [ 147.152561] Lustre: DEBUG MARKER: excepting tests: 102 106 115 32newtarball [ 147.639215] Lustre: DEBUG MARKER: skipping tests SLOW=no: 45 69 106 111 114 [ 162.271836] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 162.272114] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 162.274738] Lustre: Skipped 3 previous similar messages [ 162.276392] Lustre: Skipped 2 previous similar messages [ 165.369923] LustreError: 12120:0:(obd_class.h:479:obd_check_dev()) Device 28 not setup [ 165.395555] Lustre: server umount lustre-MDT0000 complete [ 166.654935] LustreError: 6447:0:(ldlm_lockd.c:2550:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1755039531 with bad export cookie 7654736318383941733 [ 166.656510] LustreError: MGC192.168.202.108@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 166.660146] LustreError: 6447:0:(ldlm_lockd.c:2550:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 167.391835] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 167.392103] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 167.395962] Lustre: Skipped 1 previous similar message [ 171.423558] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 171.425537] Lustre: Skipped 2 previous similar messages [ 172.793504] LustreError: 12322:0:(obd_class.h:479:obd_check_dev()) Device 27 not setup [ 172.796147] LustreError: 12322:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 172.856583] Lustre: server umount lustre-MDT0001 complete [ 174.132082] LustreError: 12523:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 174.133816] LustreError: 12523:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 174.150667] Lustre: server umount lustre-OST0000 complete [ 175.445891] Lustre: server umount lustre-OST0001 complete [ 177.431136] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing set_hostid [ 180.383996] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing load_modules_local [ 183.680203] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 185.935642] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 187.945799] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 190.092397] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 192.587940] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 192.609152] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 192.696793] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 192.709862] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 192.746596] Lustre: lustre-MDT0000: new disk, initializing [ 192.773425] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 192.780387] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 194.230384] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing set_default_debug -1 all [ 198.654782] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 198.678384] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 198.712825] Lustre: 14885:0:(mgs_llog.c:1345:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 198.821961] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 200.191249] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing set_default_debug -1 all [ 203.402307] Lustre: DEBUG MARKER: oleg208-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 204.638180] Lustre: DEBUG MARKER: oleg208-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 206.825469] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 206.848058] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 206.941645] Lustre: lustre-OST0000: new disk, initializing [ 206.942822] Lustre: Skipped 1 previous similar message [ 206.944795] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 206.947311] Lustre: Skipped 2 previous similar messages [ 206.977945] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 206.980811] Lustre: Skipped 1 previous similar message [ 208.122576] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 208.125749] Lustre: Skipped 1 previous similar message [ 208.127851] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 208.168321] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 208.844515] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing set_default_debug -1 all [ 212.005641] Lustre: DEBUG MARKER: oleg208-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 213.469040] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 213.551178] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 214.867929] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 214.948216] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 218.594036] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 218.594811] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 218.595111] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 218.597994] Lustre: Skipped 1 previous similar message [ 218.600761] LustreError: Skipped 1 previous similar message [ 221.676170] LustreError: 17790:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 221.679048] LustreError: 17790:0:(obd_class.h:479:obd_check_dev()) Skipped 9 previous similar messages [ 221.739026] Lustre: server umount lustre-OST0000 complete [ 223.712237] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 223.712862] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 223.716884] Lustre: Skipped 1 previous similar message [ 223.718312] Lustre: Skipped 1 previous similar message [ 229.114563] LustreError: 17992:0:(obd_class.h:479:obd_check_dev()) Device 22 not setup [ 229.117203] LustreError: 17992:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 229.145870] Lustre: server umount lustre-MDT0000 complete [ 230.429487] LustreError: 14878:0:(ldlm_lockd.c:2550:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1755039595 with bad export cookie 7654736318383943252 [ 230.432623] LustreError: MGC192.168.202.108@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 230.434433] LustreError: 14878:0:(ldlm_lockd.c:2550:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 230.582360] Lustre: server umount lustre-MDT0001 complete [ 232.295594] Lustre: DEBUG MARKER: == conf-sanity test 76a: set permanent params with lctl across mounts ========================================================== 18:59:58 (1755039598) [ 234.531939] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 234.685652] LustreError: 18719:0:(ldlm_lib.c:1113:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 234.691966] LustreError: 18719:0:(ldlm_lib.c:1113:target_handle_connect()) Skipped 5 previous similar messages [ 234.711116] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 236.058325] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing set_default_debug -1 all [ 238.966408] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 240.409290] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing set_default_debug -1 all [ 242.273614] Lustre: DEBUG MARKER: oleg208-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 243.473887] Lustre: DEBUG MARKER: oleg208-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 245.870733] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 247.959544] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing set_default_debug -1 all [ 249.731887] Lustre: DEBUG MARKER: oleg208-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 261.998489] Lustre: Modifying parameter general.osc.*.max_dirty_mb=452 in log params [ 269.793516] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 269.794812] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 269.796078] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 269.796084] Lustre: Skipped 3 previous similar messages [ 269.798727] Lustre: Skipped 2 previous similar messages [ 271.170052] LustreError: 21022:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup [ 271.172173] LustreError: 21022:0:(obd_class.h:479:obd_check_dev()) Skipped 13 previous similar messages [ 271.206236] Lustre: server umount lustre-MDT0000 complete [ 272.546339] LustreError: 18698:0:(ldlm_lockd.c:2550:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1755039637 with bad export cookie 7654736318383944134 [ 272.550484] LustreError: MGC192.168.202.108@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 272.550810] LustreError: 18698:0:(ldlm_lockd.c:2550:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 272.691179] Lustre: server umount lustre-MDT0001 complete [ 284.410259] Lustre: server umount lustre-OST0000 complete [ 291.906694] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing load_modules_local [ 295.657675] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 295.828261] LustreError: 22740:0:(ldlm_lib.c:1113:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 295.849586] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 295.852073] Lustre: Skipped 2 previous similar messages [ 297.210802] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing set_default_debug -1 all [ 300.272444] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 301.687024] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing set_default_debug -1 all [ 302.636621] Lustre: Setting parameter general.debug_raw_pointers=Y in log params [ 304.843077] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 306.866443] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing set_default_debug -1 all [ 309.811438] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 309.837791] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 309.907982] Lustre: lustre-OST0001: new disk, initializing [ 309.911064] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 311.229780] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 311.233925] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 311.261108] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 311.921713] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing set_default_debug -1 all [ 316.574690] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 324.220877] Lustre: Setting parameter general.obdfilter.*.client_cache_count=256 in log params [ 324.223110] Lustre: Skipped 1 previous similar message [ 330.563549] LustreError: 26258:0:(obd_class.h:479:obd_check_dev()) Device 28 not setup [ 330.566244] LustreError: 26258:0:(obd_class.h:479:obd_check_dev()) Skipped 17 previous similar messages [ 330.605075] Lustre: server umount lustre-MDT0000 complete [ 331.233227] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 331.235888] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 331.240027] LustreError: 22740:0:(ldlm_lib.c:1113:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 331.835270] LustreError: 22719:0:(ldlm_lockd.c:2550:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1755039696 with bad export cookie 7654736318383944981 [ 331.838504] LustreError: MGC192.168.202.108@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 331.839244] LustreError: 22719:0:(ldlm_lockd.c:2550:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 334.018909] Lustre: server umount lustre-MDT0001 complete [ 345.087687] Lustre: server umount lustre-OST0000 complete [ 356.663540] Lustre: server umount lustre-OST0001 complete [ 361.948027] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing load_modules_local [ 365.236661] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 365.368150] LustreError: 28027:0:(ldlm_lib.c:1113:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 365.375241] LustreError: 28027:0:(ldlm_lib.c:1113:target_handle_connect()) Skipped 3 previous similar messages [ 365.387913] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 365.390675] Lustre: Skipped 3 previous similar messages [ 366.663854] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing set_default_debug -1 all [ 369.414364] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 370.866180] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing set_default_debug -1 all [ 371.797801] Lustre: 29131:0:(mgs_llog.c:1345:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 374.014655] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 375.140719] LustreError: 29488:0:(ldlm_lib.c:1113:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 376.064858] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing set_default_debug -1 all [ 379.005861] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 380.977685] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing set_default_debug -1 all [ 384.141612] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 385.399512] Lustre: 30973:0:(mgs_llog.c:1345:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 395.231874] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 395.234346] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 395.237866] Lustre: Skipped 3 previous similar messages [ 395.239628] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 395.241237] Lustre: Skipped 2 previous similar messages [ 398.076807] LustreError: 31175:0:(obd_class.h:479:obd_check_dev()) Device 10 not setup [ 398.079147] LustreError: 31175:0:(obd_class.h:479:obd_check_dev()) Skipped 25 previous similar messages [ 398.106925] Lustre: server umount lustre-MDT0000 complete [ 399.298129] LustreError: 29134:0:(ldlm_lockd.c:2550:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1755039764 with bad export cookie 7654736318383946276 [ 399.301191] LustreError: MGC192.168.202.108@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 399.426948] Lustre: server umount lustre-MDT0001 complete [ 410.052283] Lustre: server umount lustre-OST0000 complete [ 421.685609] Lustre: server umount lustre-OST0001 complete [ 424.580777] Lustre: DEBUG MARKER: == conf-sanity test 76b: verify params log setup correctly ========================================================== 19:03:10 (1755039790) [ 433.355640] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing load_modules_local [ 436.701718] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 436.862811] LustreError: 33745:0:(ldlm_lib.c:1113:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 438.262875] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing set_default_debug -1 all [ 441.256617] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 442.768116] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing set_default_debug -1 all [ 443.764655] Lustre: 34849:0:(mgs_llog.c:1345:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 446.012538] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 448.088591] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing set_default_debug -1 all [ 450.951911] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 452.965764] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing set_default_debug -1 all [ 456.057776] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 457.231206] Lustre: 36690:0:(mgs_llog.c:1345:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 466.912175] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 466.914769] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 466.918352] Lustre: Skipped 3 previous similar messages [ 466.920663] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 466.923576] Lustre: Skipped 3 previous similar messages [ 469.693817] LustreError: 36843:0:(obd_class.h:479:obd_check_dev()) Device 10 not setup [ 469.696734] LustreError: 36843:0:(obd_class.h:479:obd_check_dev()) Skipped 25 previous similar messages [ 469.727585] Lustre: server umount lustre-MDT0000 complete [ 471.055524] LustreError: 33725:0:(ldlm_lockd.c:2550:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1755039836 with bad export cookie 7654736318383947382 [ 471.059611] LustreError: 33725:0:(ldlm_lockd.c:2550:ldlm_cancel_handler()) Skipped 5 previous similar messages [ 471.060571] LustreError: MGC192.168.202.108@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 471.208065] Lustre: server umount lustre-MDT0001 complete [ 482.820247] Lustre: server umount lustre-OST0000 complete [ 494.378505] Lustre: server umount lustre-OST0001 complete [ 497.047466] Lustre: DEBUG MARKER: == conf-sanity test 76c: verify changelog_mask is applied with lctl set_param -P ========================================================== 19:04:22 (1755039862) [ 501.961715] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing load_modules_local [ 505.260753] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 505.411533] LustreError: 38803:0:(ldlm_lib.c:1113:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 505.419611] LustreError: 38803:0:(ldlm_lib.c:1113:target_handle_connect()) Skipped 1 previous similar message [ 505.434198] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 505.436436] Lustre: Skipped 7 previous similar messages [ 506.715281] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing set_default_debug -1 all [ 509.587285] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 510.942892] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing set_default_debug -1 all [ 511.909674] Lustre: 39909:0:(mgs_llog.c:1345:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 514.144037] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 516.007240] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing set_default_debug -1 all [ 518.738774] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 520.694611] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing set_default_debug -1 all [ 523.671744] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 530.058402] Lustre: Setting parameter general.mdd.*.changelog_mask=-CLOSE in log params [ 532.030521] Lustre: lustre-MDT0000: Not available for connect from 192.168.202.8@tcp (stopping) [ 532.032490] Lustre: Skipped 4 previous similar messages [ 535.522336] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 535.523591] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 535.526231] Lustre: Skipped 4 previous similar messages [ 537.173651] Lustre: server umount lustre-MDT0000 complete [ 538.334549] LustreError: 38784:0:(ldlm_lockd.c:2550:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1755039903 with bad export cookie 7654736318383948488 [ 538.337874] LustreError: MGC192.168.202.108@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 538.339650] LustreError: 38784:0:(ldlm_lockd.c:2550:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 540.639989] LustreError: 41961:0:(ldlm_lib.c:1113:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 540.644279] LustreError: 41961:0:(ldlm_lib.c:1113:target_handle_connect()) Skipped 2 previous similar messages [ 544.523098] Lustre: server umount lustre-MDT0001 complete [ 547.385415] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 548.447114] LustreError: 42610:0:(import.c:333:ptlrpc_invalidate_import()) MGS: timeout waiting for callback (1 != 0) [ 548.450403] LustreError: 42610:0:(import.c:357:ptlrpc_invalidate_import()) @@@ still on sending list req@ffff94063b5dca80 x1840292226701952/t0(0) o250->MGC192.168.202.108@tcp@0@lo:26/25 lens 520/544 e 0 to 0 dl 1755039913 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'ptlrpcd_00_00.0' uid:0 gid:0 projid:4294967295 [ 548.457780] LustreError: 42610:0:(import.c:367:ptlrpc_invalidate_import()) MGS: Unregistering RPCs found (0). Network is sluggish? Waiting for them to error out. [ 548.767615] LustreError: 3623:0:(client.c:1385:ptlrpc_import_delay_req()) @@@ invalidate in flight req@ffff940505ffa680 x1840292226703360/t0(0) o250->MGC192.168.202.108@tcp@0@lo:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl Rpc:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 550.218254] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing set_default_debug -1 all [ 553.094217] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 554.531338] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing set_default_debug -1 all [ 556.312605] Lustre: DEBUG MARKER: oleg208-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 568.810189] Lustre: lustre-MDT0001-lwp-OST0001: Connection restored to (at 0@lo) [ 572.909992] Lustre: lustre-MDT0000-lwp-OST0001: Connection restored to (at 0@lo) [ 572.912018] Lustre: Skipped 1 previous similar message [ 573.386596] Lustre: DEBUG MARKER: mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 16 sec [ 574.523380] Lustre: DEBUG MARKER: oleg208-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 575.061891] Lustre: DEBUG MARKER: mdc.lustre-MDT0001-mdc-*.mds_server_uuid in FULL state after 0 sec [ 582.487358] Lustre: server umount lustre-MDT0000 complete [ 583.676180] LustreError: 41805:0:(ldlm_lockd.c:2550:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1755039948 with bad export cookie 7654736318383949629 [ 583.679401] LustreError: MGC192.168.202.108@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 583.680251] LustreError: 41805:0:(ldlm_lockd.c:2550:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 583.827747] Lustre: server umount lustre-MDT0001 complete [ 595.457086] Lustre: server umount lustre-OST0000 complete [ 606.898598] LustreError: 44765:0:(obd_class.h:479:obd_check_dev()) Device 2 not setup [ 606.901030] LustreError: 44765:0:(obd_class.h:479:obd_check_dev()) Skipped 61 previous similar messages [ 606.975365] Lustre: server umount lustre-OST0001 complete [ 609.746392] Lustre: DEBUG MARKER: == conf-sanity test 76d: verify llite.*.xattr_cache can be set by 'lctl set_param -P' correctly ========================================================== 19:06:15 (1755039975) [ 614.898955] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing load_modules_local [ 618.351153] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 618.487452] LustreError: 46121:0:(ldlm_lib.c:1113:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 618.494028] LustreError: 46121:0:(ldlm_lib.c:1113:target_handle_connect()) Skipped 8 previous similar messages [ 619.829463] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing set_default_debug -1 all [ 622.676052] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 624.211942] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing set_default_debug -1 all [ 625.213253] Lustre: 47228:0:(mgs_llog.c:1345:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 625.216220] Lustre: 47228:0:(mgs_llog.c:1345:mgs_modify_param()) Skipped 1 previous similar message [ 627.378812] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 629.367900] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing set_default_debug -1 all [ 632.105767] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 633.994222] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing set_default_debug -1 all [ 637.067439] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 643.565961] Lustre: Setting parameter general.llite.*.xattr_cache=0 in log params [ 648.676175] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 654.101192] Lustre: server umount lustre-MDT0000 complete [ 655.284351] LustreError: 46103:0:(ldlm_lockd.c:2550:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1755040020 with bad export cookie 7654736318383950434 [ 655.287490] LustreError: MGC192.168.202.108@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 655.288610] LustreError: 46103:0:(ldlm_lockd.c:2550:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 655.454592] Lustre: server umount lustre-MDT0001 complete [ 666.110803] Lustre: server umount lustre-OST0000 complete [ 677.569047] Lustre: server umount lustre-OST0001 complete [ 680.535575] Lustre: DEBUG MARKER: == conf-sanity test 77: comma-separated MGS NIDs and failover node NIDs ========================================================== 19:07:26 (1755040046) [ 681.328236] Lustre: DEBUG MARKER: SKIP: conf-sanity test_77 mixed loopback and real device not working [ 681.915308] Lustre: DEBUG MARKER: == conf-sanity test 78: run resize2fs on MDT and OST filesystems ========================================================== 19:07:27 (1755040047) [ 685.078387] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing set_hostid [ 687.705913] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing load_modules_local [ 690.932724] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 692.880241] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 695.149224] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 695.170193] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 695.265581] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 695.302346] Lustre: lustre-MDT0000: new disk, initializing [ 695.333291] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 696.537215] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing set_default_debug -1 all [ 699.519826] Lustre: DEBUG MARKER: oleg208-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 701.762261] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 701.783676] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 703.043791] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 703.054087] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 703.757513] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing set_default_debug -1 all [ 706.784271] Lustre: DEBUG MARKER: oleg208-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 707.459424] Lustre: DEBUG MARKER: create test files [ 718.305709] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 718.305925] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 718.310239] Lustre: Skipped 12 previous similar messages [ 718.314809] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 718.316382] Lustre: Skipped 22 previous similar messages [ 722.803174] Lustre: server umount lustre-OST0000 complete [ 724.161483] Lustre: server umount lustre-MDT0000 complete [ 728.624863] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing unload_modules_local [ 729.703112] Key type lgssc unregistered [ 729.836803] LNet: 54568:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 730.853281] LNet: Removed LNI 192.168.202.108@tcp [ 731.178155] Key type .llcrypt unregistered [ 731.179509] Key type ._llcrypt unregistered [ 741.383652] Key type ._llcrypt registered [ 741.384687] Key type .llcrypt registered [ 741.422802] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing load_modules_local [ 741.894942] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 741.930235] alg: No test for adler32 (adler32-zlib) [ 742.781810] Lustre: Lustre: Build Version: 2.16.56_30_ga643f08 [ 742.856640] LNet: Added LNI 192.168.202.108@tcp [8/256/0/180] [ 742.858779] LNet: Accept secure, port 988 [ 744.439103] Key type lgssc registered [ 744.741482] Lustre: Echo OBD driver; http://www.lustre.org/ [ 747.202134] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 747.211895] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 748.374563] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 749.593399] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing set_default_debug -1 all [ 751.305196] Lustre: DEBUG MARKER: oleg208-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 753.593895] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 753.733946] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 755.621430] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing set_default_debug -1 all [ 757.286140] Lustre: DEBUG MARKER: oleg208-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 758.883063] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:102 to 0x240000400:129) [ 758.907015] Lustre: 57676:0:(osd_internal.h:1335:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 515 < left 544, rollback = 9 [ 758.910072] Lustre: 57676:0:(osd_handler.c:1966:osd_trans_dump_creds()) create: 32/128/0, destroy: 0/0/0 [ 758.912165] Lustre: 57676:0:(osd_handler.c:1973:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 758.914746] Lustre: 57676:0:(osd_handler.c:1983:osd_trans_dump_creds()) write: 2/13/1, punch: 0/0/0, quota 1/3/0 [ 758.916952] Lustre: 57676:0:(osd_handler.c:1990:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 758.919114] Lustre: 57676:0:(osd_handler.c:1997:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 765.750774] Lustre: DEBUG MARKER: check files after expanding the MDT and OST filesystems [ 767.309761] Lustre: DEBUG MARKER: create more files after expanding the MDT and OST filesystems [ 769.504927] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 769.505061] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 769.512385] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 774.623930] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 775.086062] LustreError: 57865:0:(obd_class.h:479:obd_check_dev()) Device 13 not setup [ 775.147104] Lustre: server umount lustre-OST0000 complete [ 776.358651] LustreError: 58067:0:(obd_class.h:479:obd_check_dev()) Device 8 not setup [ 776.360464] LustreError: 58067:0:(obd_class.h:479:obd_check_dev()) Skipped 1 previous similar message [ 776.457968] Lustre: server umount lustre-MDT0000 complete [ 781.032671] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing unload_modules_local [ 782.183516] Key type lgssc unregistered [ 782.330519] LNet: 58594:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 783.397514] LNet: Removed LNI 192.168.202.108@tcp [ 783.722881] Key type .llcrypt unregistered [ 783.723981] Key type ._llcrypt unregistered [ 794.445482] Key type ._llcrypt registered [ 794.446387] Key type .llcrypt registered [ 794.482169] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing load_modules_local [ 794.953315] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 794.962537] alg: No test for adler32 (adler32-zlib) [ 795.815346] Lustre: Lustre: Build Version: 2.16.56_30_ga643f08 [ 795.898225] LNet: Added LNI 192.168.202.108@tcp [8/256/0/180] [ 795.900308] LNet: Accept secure, port 988 [ 797.487230] Key type lgssc registered [ 797.893709] Lustre: Echo OBD driver; http://www.lustre.org/ [ 800.624494] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 800.632826] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 801.805730] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 803.132646] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing set_default_debug -1 all [ 804.913910] Lustre: DEBUG MARKER: oleg208-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 807.251391] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 807.378475] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 809.353451] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing set_default_debug -1 all [ 811.307369] Lustre: DEBUG MARKER: oleg208-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 811.494435] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:140 to 0x240000400:161) [ 811.516357] Lustre: 61257:0:(osd_internal.h:1335:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 515 < left 544, rollback = 9 [ 811.519270] Lustre: 61257:0:(osd_handler.c:1966:osd_trans_dump_creds()) create: 32/128/0, destroy: 0/0/0 [ 811.521826] Lustre: 61257:0:(osd_handler.c:1973:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 811.524602] Lustre: 61257:0:(osd_handler.c:1983:osd_trans_dump_creds()) write: 2/13/1, punch: 0/0/0, quota 1/3/0 [ 811.527549] Lustre: 61257:0:(osd_handler.c:1990:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 811.529590] Lustre: 61257:0:(osd_handler.c:1997:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 819.567031] Lustre: DEBUG MARKER: check files after shrinking the MDT and OST filesystems [ 826.848522] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 826.849267] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 826.855662] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 827.820229] LustreError: 61943:0:(obd_class.h:479:obd_check_dev()) Device 13 not setup [ 827.885872] Lustre: server umount lustre-OST0000 complete [ 829.108381] LustreError: 62145:0:(obd_class.h:479:obd_check_dev()) Device 8 not setup [ 829.110546] LustreError: 62145:0:(obd_class.h:479:obd_check_dev()) Skipped 1 previous similar message [ 829.195355] Lustre: server umount lustre-MDT0000 complete [ 833.581947] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing unload_modules_local [ 834.709754] Key type lgssc unregistered [ 834.853345] LNet: 62672:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 835.877406] LNet: Removed LNI 192.168.202.108@tcp [ 836.205186] Key type .llcrypt unregistered [ 836.206396] Key type ._llcrypt unregistered [ 841.918096] Key type ._llcrypt registered [ 841.919651] Key type .llcrypt registered [ 841.954845] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing set_hostid [ 848.280423] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing load_modules_local [ 848.577686] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 848.593280] alg: No test for adler32 (adler32-zlib) [ 849.446799] Lustre: Lustre: Build Version: 2.16.56_30_ga643f08 [ 849.526860] LNet: Added LNI 192.168.202.108@tcp [8/256/0/180] [ 849.528790] LNet: Accept secure, port 988 [ 851.103146] Key type lgssc registered [ 851.435625] Lustre: Echo OBD driver; http://www.lustre.org/ [ 854.383819] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 856.511165] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 858.478980] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 860.492131] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 862.715669] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 862.733079] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 862.739565] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 863.824190] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 863.838259] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 863.872550] Lustre: lustre-MDT0000: new disk, initializing [ 863.896555] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 863.903396] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 865.179546] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing set_default_debug -1 all [ 869.204716] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 869.226135] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 869.259026] Lustre: 65756:0:(mgs_llog.c:1345:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 869.282470] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 869.284295] Lustre: Skipped 1 previous similar message [ 869.320229] Lustre: lustre-MDT0001: new disk, initializing [ 869.360080] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 869.373864] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 869.378786] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 870.560621] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing set_default_debug -1 all [ 873.502250] Lustre: DEBUG MARKER: oleg208-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 874.599431] Lustre: DEBUG MARKER: oleg208-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 876.765903] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 876.784961] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 876.883492] Lustre: lustre-OST0000: new disk, initializing [ 876.885617] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 876.916927] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 878.766710] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing set_default_debug -1 all [ 880.701276] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 880.705847] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 880.747437] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 881.747578] Lustre: DEBUG MARKER: oleg208-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 883.094947] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 883.171253] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 884.508846] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 884.598705] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 885.731145] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 885.731839] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 885.732819] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 885.735027] Lustre: Skipped 1 previous similar message [ 887.276347] LustreError: 68662:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 887.346479] Lustre: server umount lustre-OST0000 complete [ 889.823817] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 889.826852] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 889.831958] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 889.833869] Lustre: Skipped 1 previous similar message [ 890.848354] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 890.848745] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 894.714898] LustreError: 68863:0:(obd_class.h:479:obd_check_dev()) Device 22 not setup [ 894.716812] LustreError: 68863:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 894.745636] Lustre: server umount lustre-MDT0000 complete [ 895.953575] LustreError: 66529:0:(ldlm_lockd.c:2550:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1755040261 with bad export cookie 7238542811311555084 [ 895.956503] LustreError: MGC192.168.202.108@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 895.957765] LustreError: 66529:0:(ldlm_lockd.c:2550:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 896.019244] LustreError: 69064:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 896.021777] LustreError: 69064:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 896.109141] Lustre: server umount lustre-MDT0001 complete [ 898.900148] Lustre: DEBUG MARKER: == conf-sanity test 79: format MDT/OST without mgs option (should return errors) ========================================================== 19:11:04 (1755040264) [ 902.312848] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing load_modules_local [ 906.916129] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 914.886325] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing set_hostid [ 917.910970] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing load_modules_local [ 921.316647] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 923.548597] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 925.511957] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 927.590147] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 929.903889] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 929.924192] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 930.003901] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 930.015920] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 930.051850] Lustre: lustre-MDT0000: new disk, initializing [ 930.075917] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 930.082602] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 931.370482] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing set_default_debug -1 all [ 935.564691] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 935.589043] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 935.622659] Lustre: 73270:0:(mgs_llog.c:1345:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 935.647063] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 935.649495] Lustre: Skipped 1 previous similar message [ 935.684888] Lustre: lustre-MDT0001: new disk, initializing [ 935.718358] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 935.730632] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 935.734424] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 937.056273] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing set_default_debug -1 all [ 940.167797] Lustre: DEBUG MARKER: oleg208-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 941.387568] Lustre: DEBUG MARKER: oleg208-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 943.675955] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 943.700717] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 943.795097] Lustre: lustre-OST0000: new disk, initializing [ 943.797554] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 943.827304] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 945.718780] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing set_default_debug -1 all [ 948.952563] Lustre: DEBUG MARKER: oleg208-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 949.239577] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 949.242946] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 949.274251] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 950.259943] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 950.332619] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 951.591426] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 951.665377] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 954.337937] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 954.338496] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 954.339328] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 954.339332] Lustre: Skipped 1 previous similar message [ 954.340799] LustreError: Skipped 1 previous similar message [ 954.348696] Lustre: Skipped 1 previous similar message [ 958.379101] LustreError: 76175:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 958.381099] LustreError: 76175:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 958.443575] Lustre: server umount lustre-OST0000 complete [ 961.503925] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 961.507542] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 961.513913] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 961.515733] Lustre: Skipped 1 previous similar message [ 965.756167] LustreError: 76376:0:(obd_class.h:479:obd_check_dev()) Device 22 not setup [ 965.759191] LustreError: 76376:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 965.788959] Lustre: server umount lustre-MDT0000 complete [ 967.096205] LustreError: 75002:0:(ldlm_lockd.c:2550:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1755040332 with bad export cookie 7238542811311555952 [ 967.099532] LustreError: 75002:0:(ldlm_lockd.c:2550:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 967.099821] LustreError: MGC192.168.202.108@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 967.243802] Lustre: server umount lustre-MDT0001 complete [ 970.036390] Lustre: DEBUG MARKER: == conf-sanity test 80: mgc import reconnect race ======== 19:12:15 (1755040335) [ 972.323133] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 972.514118] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 973.878381] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing set_default_debug -1 all [ 976.797000] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 978.238559] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing set_default_debug -1 all [ 980.147267] Lustre: DEBUG MARKER: oleg208-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 981.422089] Lustre: DEBUG MARKER: oleg208-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 983.772422] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 985.952460] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing set_default_debug -1 all [ 987.827843] Lustre: DEBUG MARKER: oleg208-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 989.087819] Lustre: 79568:0:(genops.c:1791:obd_export_evict_by_uuid()) MGS: evicting 3d0c367a-682f-4436-97c9-f6d7f47f415b at adminstrative request [ 992.227259] LustreError: MGC192.168.202.108@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 992.232413] Lustre: *** cfs_fail_loc=906, val=2147483648*** [ 1002.469873] Lustre: MGS: Client 3d0c367a-682f-4436-97c9-f6d7f47f415b (at 0@lo) reconnecting [ 1002.472938] Lustre: Evicted from MGS (at 192.168.202.108@tcp) after server handle changed from 0x64747855dfaec8d4 to 0x64747855dfaecb5f [ 1002.479416] Lustre: MGC192.168.202.108@tcp: Connection restored to 192.168.202.108@tcp (at 0@lo) [ 1021.611423] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1021.648474] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1021.713096] Lustre: lustre-OST0001: new disk, initializing [ 1021.715122] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 1021.749863] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 1021.752718] Lustre: Skipped 2 previous similar messages [ 1023.634227] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing set_default_debug -1 all [ 1026.841550] Lustre: DEBUG MARKER: oleg208-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid [ 1027.850075] LustreError: 80686:0:(obd_class.h:479:obd_check_dev()) Device 25 not setup [ 1027.852178] LustreError: 80686:0:(obd_class.h:479:obd_check_dev()) Skipped 13 previous similar messages [ 1027.869836] Lustre: server umount lustre-OST0001 complete [ 1029.128762] LustreError: 78834:0:(ldlm_lib.c:1113:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1029.135708] LustreError: 78834:0:(ldlm_lib.c:1113:target_handle_connect()) Skipped 2 previous similar messages [ 1029.136138] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1029.144250] Lustre: Skipped 1 previous similar message [ 1029.147244] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1029.150100] Lustre: Skipped 2 previous similar messages [ 1034.210313] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1034.211488] LustreError: 78835:0:(ldlm_lib.c:1113:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1034.217383] LustreError: 78835:0:(ldlm_lib.c:1113:target_handle_connect()) Skipped 2 previous similar messages [ 1035.249093] Lustre: server umount lustre-OST0000 complete [ 1042.711649] Lustre: server umount lustre-MDT0000 complete [ 1043.974281] LustreError: 77229:0:(ldlm_lockd.c:2550:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1755040409 with bad export cookie 7238542811311557471 [ 1043.977751] LustreError: 77229:0:(ldlm_lockd.c:2550:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 1043.978245] LustreError: MGC192.168.202.108@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1044.048988] LustreError: 81300:0:(obd_class.h:479:obd_check_dev()) Device 23 not setup [ 1044.050750] LustreError: 81300:0:(obd_class.h:479:obd_check_dev()) Skipped 15 previous similar messages [ 1044.145315] Lustre: server umount lustre-MDT0001 complete [ 1046.922897] Lustre: DEBUG MARKER: == conf-sanity test 81: sparse OST indexing ============== 19:13:32 (1755040412) [ 1047.418619] Lustre: DEBUG MARKER: SKIP: conf-sanity test_81 needs >= 3 OSTs [ 1047.987841] Lustre: DEBUG MARKER: == conf-sanity test 82a: specify OSTs for file (succeed) or directory (succeed) ========================================================== 19:13:33 (1755040413) [ 1048.523883] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82a needs >= 3 OSTs [ 1049.144035] Lustre: DEBUG MARKER: == conf-sanity test 82b: specify OSTs for file with --pool and --ost-list options ========================================================== 19:13:34 (1755040414) [ 1049.724526] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82b needs >= 4 OSTs [ 1050.314785] Lustre: DEBUG MARKER: == conf-sanity test 83: ENOSPACE on OST doesn't cause message VFS: Busy inodes after unmount ... ========================================================== 19:13:36 (1755040416) [ 1050.841316] Lustre: DEBUG MARKER: mount the OST /dev/mapper/ost1_flakey as a ldiskfs filesystem [ 1052.565713] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1052.925291] loop: module loaded [ 1052.929050] loop0: detected capacity change from 0 to 4294967296 [ 1052.946594] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 1053.499208] Lustre: DEBUG MARKER: run llverfs in partial mode on the OST ldiskfs /mnt/lustre-ost1 [ 1054.562275] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing run_llverfs /mnt/lustre-ost1 -vpl no [ 1056.026255] Lustre: DEBUG MARKER: unmount the OST /dev/mapper/ost1_flakey [ 1058.862131] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1060.141353] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1061.477642] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1062.827820] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1065.036656] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1065.058548] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1065.068521] LustreError: 83861:0:(obd_config.c:779:class_setup()) setup lustre-OST0000-osd failed (-28) [ 1065.071047] LustreError: 83861:0:(obd_mount.c:193:lustre_start_simple()) lustre-OST0000-osd setup error -28 [ 1065.073855] LustreError: 83861:0:(tgt_mount.c:2240:server_fill_super()) Unable to start osd on /dev/mapper/ost1_flakey: -28 [ 1065.078422] LustreError: 83861:0:(super25.c:181:lustre_fill_super()) llite: Unable to mount : rc = -28 [ 1071.276056] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing set_hostid [ 1074.340991] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing load_modules_local [ 1077.830585] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1080.061663] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1082.106957] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1084.154897] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1086.447176] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1086.470568] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1086.557590] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 1086.570572] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1086.609107] Lustre: lustre-MDT0000: new disk, initializing [ 1086.631431] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1086.638487] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1088.006925] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing set_default_debug -1 all [ 1092.166817] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1092.186765] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1092.217169] Lustre: 86682:0:(mgs_llog.c:1345:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 1092.321316] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 1093.667221] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing set_default_debug -1 all [ 1096.779754] Lustre: DEBUG MARKER: oleg208-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1098.021799] Lustre: DEBUG MARKER: oleg208-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 1100.384342] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1100.410309] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1102.548339] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing set_default_debug -1 all [ 1105.548138] Lustre: DEBUG MARKER: oleg208-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1106.853591] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 1109.494820] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 1109.529509] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 1109.991372] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 3 sec [ 1111.304446] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 1111.393514] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 1114.596432] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1114.596626] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 1114.599780] Lustre: Skipped 3 previous similar messages [ 1114.600466] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1114.602258] LustreError: Skipped 2 previous similar messages [ 1114.608771] Lustre: Skipped 6 previous similar messages [ 1118.123407] LustreError: 89608:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 1118.125443] LustreError: 89608:0:(obd_class.h:479:obd_check_dev()) Skipped 9 previous similar messages [ 1118.197463] Lustre: server umount lustre-OST0000 complete [ 1125.648677] Lustre: server umount lustre-MDT0000 complete [ 1126.892569] LustreError: 89811:0:(ldlm_lockd.c:2550:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1755040492 with bad export cookie 7238542811311558059 [ 1126.897432] LustreError: 89811:0:(ldlm_lockd.c:2550:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1126.898036] LustreError: MGC192.168.202.108@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1127.051672] Lustre: server umount lustre-MDT0001 complete [ 1129.913237] Lustre: DEBUG MARKER: == conf-sanity test 84: check recovery_hard_time ========= 19:14:55 (1755040495) [ 1132.248087] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1132.400072] LustreError: 90683:0:(ldlm_lib.c:1113:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1133.807107] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing set_default_debug -1 all [ 1136.855793] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1138.349565] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing set_default_debug -1 all [ 1140.257031] Lustre: DEBUG MARKER: oleg208-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1141.537520] Lustre: DEBUG MARKER: oleg208-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 1143.929357] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1146.039206] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing set_default_debug -1 all [ 1147.887392] Lustre: DEBUG MARKER: oleg208-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1149.279121] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 1157.532461] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 8 sec [ 1158.847323] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 1158.926126] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 1161.206916] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1161.232930] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1161.287025] Lustre: lustre-OST0001: new disk, initializing [ 1161.288777] Lustre: Skipped 2 previous similar messages [ 1161.291788] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 1161.293702] Lustre: Skipped 3 previous similar messages [ 1163.425993] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing set_default_debug -1 all [ 1166.600167] Lustre: DEBUG MARKER: oleg208-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid [ 1168.054809] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid 50 [ 1170.428160] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 1170.430812] Lustre: Skipped 2 previous similar messages [ 1170.432784] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 1170.459226] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 1171.206388] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid in FULL state after 3 sec [ 1172.678274] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [ 1172.770693] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 1182.682807] Lustre: DEBUG MARKER: mds1 REPLAY BARRIER on lustre-MDT0000 [ 1187.920655] Lustre: Failing over lustre-MDT0000 [ 1188.217192] LustreError: 95595:0:(obd_class.h:479:obd_check_dev()) Device 28 not setup [ 1188.218890] LustreError: 95595:0:(obd_class.h:479:obd_check_dev()) Skipped 17 previous similar messages [ 1188.250535] Lustre: server umount lustre-MDT0000 complete [ 1188.320806] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1188.323410] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1188.327693] Lustre: Skipped 3 previous similar messages [ 1188.330144] LustreError: 90682:0:(ldlm_lib.c:1113:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1194.553506] LustreError: 90677:0:(ldlm_lib.c:1113:target_handle_connect()) lustre-MDT0000: not available for connect from 192.168.202.8@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1194.562391] LustreError: 90677:0:(ldlm_lib.c:1113:target_handle_connect()) Skipped 7 previous similar messages [ 1203.045632] LDISKFS-fs (dm-0): recovery complete [ 1203.047041] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1203.089209] LustreError: MGC192.168.202.108@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1203.204853] Lustre: lustre-MDT0000: in recovery but waiting for the first client to connect [ 1204.481340] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing set_default_debug -1 all [ 1204.794538] Lustre: lustre-MDT0000: Will be in recovery for at least 1:00, or until 3 clients reconnect [ 1204.794611] Lustre: 90679:0:(ldlm_lib.c:2001:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 1208.289835] Lustre: 90682:0:(ldlm_lib.c:2001:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 1208.292880] Lustre: 90682:0:(ldlm_lib.c:2001:extend_recovery_timer()) Skipped 1 previous similar message [ 1208.299318] Lustre: lustre-MDT0000-lwp-OST0001: Connection restored to (at 0@lo) [ 1208.317952] LustreError: 96340:0:(ldlm_lib.c:2613:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 1208.631095] LustreError: 96340:0:(ldlm_lib.c:2613:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 1208.945610] LustreError: 96340:0:(ldlm_lib.c:2613:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 1208.948327] LustreError: 96340:0:(ldlm_lib.c:2613:replay_request_or_update()) Skipped 1 previous similar message [ 1209.263162] LustreError: 96340:0:(ldlm_lib.c:2613:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 1209.265215] LustreError: 96340:0:(ldlm_lib.c:2613:replay_request_or_update()) Skipped 1 previous similar message [ 1209.585234] Lustre: 96340:0:(ldlm_lib.c:2001:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 1209.588205] Lustre: 96340:0:(ldlm_lib.c:2001:extend_recovery_timer()) Skipped 8 previous similar messages [ 1210.217397] LustreError: 96340:0:(ldlm_lib.c:2613:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 1210.220486] LustreError: 96340:0:(ldlm_lib.c:2613:replay_request_or_update()) Skipped 3 previous similar messages [ 1210.535104] LustreError: 96340:0:(ldlm_lib.c:2613:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 1210.537214] LustreError: 96340:0:(ldlm_lib.c:2613:replay_request_or_update()) Skipped 3 previous similar messages [ 1211.791936] Lustre: 96340:0:(ldlm_lib.c:2001:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 1211.796803] Lustre: 96340:0:(ldlm_lib.c:2001:extend_recovery_timer()) Skipped 6 previous similar messages [ 1212.424163] LustreError: 96340:0:(ldlm_lib.c:2613:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 1212.427911] LustreError: 96340:0:(ldlm_lib.c:2613:replay_request_or_update()) Skipped 6 previous similar messages [ 1212.743149] LustreError: 96340:0:(ldlm_lib.c:2613:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 1212.746800] LustreError: 96340:0:(ldlm_lib.c:2613:replay_request_or_update()) Skipped 6 previous similar messages [ 1215.873367] Lustre: 96340:0:(ldlm_lib.c:2001:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 1215.877725] Lustre: 96340:0:(ldlm_lib.c:2001:extend_recovery_timer()) Skipped 12 previous similar messages [ 1216.505049] LustreError: 96340:0:(ldlm_lib.c:2613:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 1216.507879] LustreError: 96340:0:(ldlm_lib.c:2613:replay_request_or_update()) Skipped 12 previous similar messages [ 1216.823122] LustreError: 96340:0:(ldlm_lib.c:2613:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 1216.825419] LustreError: 96340:0:(ldlm_lib.c:2613:replay_request_or_update()) Skipped 12 previous similar messages [ 1224.009134] Lustre: 96340:0:(ldlm_lib.c:2001:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 1224.012906] Lustre: 96340:0:(ldlm_lib.c:2001:extend_recovery_timer()) Skipped 25 previous similar messages [ 1224.641358] LustreError: 96340:0:(ldlm_lib.c:2613:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 1224.644335] LustreError: 96340:0:(ldlm_lib.c:2613:replay_request_or_update()) Skipped 25 previous similar messages [ 1224.959096] LustreError: 96340:0:(ldlm_lib.c:2613:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 1224.962500] LustreError: 96340:0:(ldlm_lib.c:2613:replay_request_or_update()) Skipped 25 previous similar messages [ 1240.265358] Lustre: 96340:0:(ldlm_lib.c:2001:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 1240.269164] Lustre: 96340:0:(ldlm_lib.c:2001:extend_recovery_timer()) Skipped 51 previous similar messages [ 1240.897105] LustreError: 96340:0:(ldlm_lib.c:2613:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 1240.900255] LustreError: 96340:0:(ldlm_lib.c:2613:replay_request_or_update()) Skipped 51 previous similar messages [ 1241.215126] LustreError: 96340:0:(ldlm_lib.c:2613:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 1241.219281] LustreError: 96340:0:(ldlm_lib.c:2613:replay_request_or_update()) Skipped 51 previous similar messages [ 1264.311116] Lustre: 96340:0:(ldlm_lib.c:2309:target_recovery_overseer()) lustre-MDT0000 recovery is aborted by hard timeout [ 1264.314284] Lustre: 96340:0:(ldlm_lib.c:2319:target_recovery_overseer()) recovery is aborted, evict exports in recovery [ 1264.316525] Lustre: 96340:0:(genops.c:1618:class_disconnect_stale_exports()) lustre-MDT0000: disconnect stale client 1598f658-d961-484b-b9f0-637dc2a20c4d@192.168.202.8@tcp [ 1264.319433] Lustre: lustre-MDT0000: disconnecting 1 stale clients [ 1264.321464] Lustre: 96340:0:(ldlm_lib.c:1831:abort_req_replay_queue()) @@@ aborted: req@00000000af3d0c41 x1840293042260096/t0(8589934776) o35->1598f658-d961-484b-b9f0-637dc2a20c4d@192.168.202.8@tcp:105/0 lens 392/0 e 0 to 0 dl 1755040640 ref 1 fl Complete:/604/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:0 [ 1264.332275] Lustre: lustre-MDT0000-osd: cancel update llog [0x200000400:0x1:0x0] [ 1264.337542] Lustre: lustre-MDT0000: Denying connection for new client 1598f658-d961-484b-b9f0-637dc2a20c4d (at 192.168.202.8@tcp), waiting for 3 known clients (2 recovered, 0 in progress, and 1 evicted) already passed deadline 0:00 [ 1264.340831] Lustre: lustre-MDT0001-osp-MDT0000: cancel update llog [0x240000401:0x1:0x0] [ 1264.355993] Lustre: lustre-MDT0000-osp-MDT0001: Connection restored to (at 0@lo) [ 1264.356368] Lustre: lustre-MDT0000: Recovery over after 1:00, of 3 clients 2 recovered and 1 was evicted. [ 1264.358786] Lustre: Skipped 2 previous similar messages [ 1264.378300] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:79 to 0x280000401:577) [ 1264.378641] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:47 to 0x2c0000401:545) [ 1274.854429] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1274.856090] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1274.858148] Lustre: Skipped 4 previous similar messages [ 1274.862114] Lustre: Skipped 5 previous similar messages [ 1277.370691] Lustre: server umount lustre-OST0000 complete [ 1279.971066] LustreError: 92269:0:(ldlm_lib.c:1113:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1279.975347] LustreError: 92269:0:(ldlm_lib.c:1113:target_handle_connect()) Skipped 11 previous similar messages [ 1284.818638] Lustre: server umount lustre-OST0001 complete [ 1290.208946] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1292.441342] Lustre: server umount lustre-MDT0000 complete [ 1293.629152] LustreError: 90663:0:(ldlm_lockd.c:2550:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1755040658 with bad export cookie 7238542811311609243 [ 1293.632397] LustreError: 90663:0:(ldlm_lockd.c:2550:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 1293.633335] LustreError: MGC192.168.202.108@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1293.780685] Lustre: server umount lustre-MDT0001 complete [ 1296.653158] Lustre: DEBUG MARKER: == conf-sanity test 85: osd_ost init: fail ea_fid_set ==== 19:17:42 (1755040662) [ 1299.160886] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1299.164325] Lustre: *** cfs_fail_loc=197, val=0*** [ 1316.639498] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 1322.783349] LustreError: 98289:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.202.108@tcp: failed processing log, type 4: rc = -110 [ 1328.934661] Lustre: *** cfs_fail_loc=197, val=0*** [ 1328.935784] Lustre: Skipped 65 previous similar messages [ 1351.455305] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1351.457443] Lustre: Skipped 7 previous similar messages [ 1353.215453] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing set_default_debug -1 all [ 1355.006943] Lustre: DEBUG MARKER: oleg208-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1355.716225] LustreError: 98947:0:(obd_class.h:479:obd_check_dev()) Device 1 not setup [ 1355.718070] LustreError: 98947:0:(obd_class.h:479:obd_check_dev()) Skipped 33 previous similar messages [ 1355.787186] Lustre: server umount lustre-OST0000 complete [ 1358.481938] Lustre: DEBUG MARKER: == conf-sanity test 86: Replacing mkfs.lustre -G option == 19:18:44 (1755040724) [ 1372.125220] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1372.315584] LustreError: 99793:0:(ldlm_lib.c:1113:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1373.658821] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing set_default_debug -1 all [ 1376.829827] Lustre: DEBUG MARKER: oleg208-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1378.890863] LustreError: 99825:0:(osp_object.c:617:osp_attr_get()) lustre-MDT0001-osp-MDT0000: osp_attr_get update error [0x200000009:0x1:0x0]: rc = -5 [ 1378.894837] LustreError: 99825:0:(lod_sub_object.c:917:lod_sub_prep_llog()) lustre-MDT0000-mdtlov: can't get id from catalogs: rc = -5 [ 1378.897061] LustreError: 99825:0:(lod_dev.c:510:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 7, retries 0, failed: rc = -5 [ 1379.060523] Lustre: server umount lustre-MDT0000 complete [ 1384.461991] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1390.580424] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing set_hostid [ 1393.744196] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing load_modules_local [ 1397.448513] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1399.969057] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1402.629293] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1404.920561] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1407.275320] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1407.297614] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1407.390889] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 1407.402137] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1407.438464] Lustre: lustre-MDT0000: new disk, initializing [ 1407.467914] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1408.790206] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing set_default_debug -1 all [ 1413.061589] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1413.092275] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1413.127130] Lustre: 103836:0:(mgs_llog.c:1345:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 1413.249917] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 1414.628289] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing set_default_debug -1 all [ 1417.790046] Lustre: DEBUG MARKER: oleg208-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1419.010895] Lustre: DEBUG MARKER: oleg208-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 1421.326432] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1421.347524] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1423.341707] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 1423.441991] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing set_default_debug -1 all [ 1426.930702] Lustre: DEBUG MARKER: oleg208-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1428.315611] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 1428.389221] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 1429.676808] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 1429.767531] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 1433.570290] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1433.570560] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 1433.571510] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1433.571515] Lustre: Skipped 4 previous similar messages [ 1433.573955] Lustre: Skipped 5 previous similar messages [ 1433.576519] LustreError: Skipped 1 previous similar message [ 1436.656235] Lustre: server umount lustre-OST0000 complete [ 1444.065172] Lustre: server umount lustre-MDT0000 complete [ 1445.611634] LustreError: 104610:0:(ldlm_lockd.c:2550:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1755040810 with bad export cookie 7238542811311619883 [ 1445.616658] LustreError: MGC192.168.202.108@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1445.619155] LustreError: 104610:0:(ldlm_lockd.c:2550:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1445.779375] Lustre: server umount lustre-MDT0001 complete [ 1449.142747] Lustre: DEBUG MARKER: == conf-sanity test 87: check if MDT inode can hold EAs with N stripes properly ========================================================== 19:20:14 (1755040814) [ 1453.477527] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing unload_modules_local [ 1454.651057] Key type lgssc unregistered [ 1454.794378] LNet: 107868:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1455.845749] LNet: Removed LNI 192.168.202.108@tcp [ 1456.361176] Key type .llcrypt unregistered [ 1456.363210] Key type ._llcrypt unregistered [ 1462.969492] Key type ._llcrypt registered [ 1462.970590] Key type .llcrypt registered [ 1463.014464] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing set_hostid [ 1469.262148] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing load_modules_local [ 1469.607756] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1469.621340] alg: No test for adler32 (adler32-zlib) [ 1470.513785] Lustre: Lustre: Build Version: 2.16.56_30_ga643f08 [ 1470.625443] LNet: Added LNI 192.168.202.108@tcp [8/256/0/180] [ 1470.627362] LNet: Accept secure, port 988 [ 1472.223136] Key type lgssc registered [ 1472.574781] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1475.969589] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1478.586728] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1480.767205] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1482.827909] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1484.699942] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1487.094256] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1487.132252] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1487.140986] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1488.242597] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 1488.256818] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1488.296581] Lustre: lustre-MDT0000: new disk, initializing [ 1488.324081] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1488.330757] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1489.692574] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing set_default_debug -1 all [ 1494.797676] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1494.839792] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1494.995634] Lustre: lustre-OST0000: new disk, initializing [ 1494.999632] Lustre: lustre-OST0000: Not available for connect from 0@lo (not set up) [ 1494.999835] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1495.006944] Lustre: Skipped 1 previous similar message [ 1495.036375] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1497.392778] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing set_default_debug -1 all [ 1500.160723] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 1500.163393] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 1500.173741] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 1502.156914] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1502.189282] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1502.261741] Lustre: lustre-OST0001: new disk, initializing [ 1502.265740] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 1502.303504] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 1504.428951] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing set_default_debug -1 all [ 1505.608399] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:1:ost [ 1505.611397] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:1:ost] [ 1505.621360] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x280000400 [ 1510.882539] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1510.883771] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1510.892144] Lustre: Skipped 1 previous similar message [ 1515.022208] LustreError: 113389:0:(obd_class.h:479:obd_check_dev()) Device 17 not setup [ 1515.127779] Lustre: server umount lustre-MDT0000 complete [ 1517.894154] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: (null) [ 1519.744852] LustreError: 111161:0:(ldlm_lockd.c:2550:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1755040884 with bad export cookie 313984267822972752 [ 1519.747691] LustreError: MGC192.168.202.108@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1519.749211] LustreError: 111161:0:(ldlm_lockd.c:2550:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 1519.768121] LustreError: 113982:0:(obd_class.h:479:obd_check_dev()) Device 13 not setup [ 1519.769979] LustreError: 113982:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 1519.785556] Lustre: server umount lustre-OST0000 complete [ 1521.025133] LustreError: 114182:0:(obd_class.h:479:obd_check_dev()) Device 2 not setup [ 1521.028076] LustreError: 114182:0:(obd_class.h:479:obd_check_dev()) Skipped 1 previous similar message [ 1521.106179] Lustre: server umount lustre-OST0001 complete [ 1524.303314] Lustre: DEBUG MARKER: == conf-sanity test 88: check the default mount options can be overridden ========================================================== 19:21:30 (1755040890) [ 1528.052873] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing load_modules_local [ 1531.216245] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1531.563533] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1533.390539] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1533.741258] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1535.984147] Lustre: DEBUG MARKER: == conf-sanity test 89: check tunefs --param and --erase-param{s} options ========================================================== 19:21:41 (1755040901) [ 1540.180992] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1540.869049] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1541.523453] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1542.179969] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1542.518659] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1543.185451] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1548.675276] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing set_hostid [ 1551.849482] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing load_modules_local [ 1555.159169] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1557.399607] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1559.430619] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1561.558830] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1563.784663] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1563.803580] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1563.890857] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 1563.903440] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1563.940611] Lustre: lustre-MDT0000: new disk, initializing [ 1563.965773] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1563.971496] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1565.306252] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing set_default_debug -1 all [ 1569.863400] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1569.886906] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1569.918260] Lustre: 119468:0:(mgs_llog.c:1345:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 1569.946455] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 1569.948217] Lustre: Skipped 1 previous similar message [ 1569.985040] Lustre: lustre-MDT0001: new disk, initializing [ 1570.024501] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 1570.038769] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 1570.041832] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 1571.470911] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing set_default_debug -1 all [ 1574.672556] Lustre: DEBUG MARKER: oleg208-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1576.017925] Lustre: DEBUG MARKER: oleg208-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 1578.313147] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1578.339711] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1578.453101] Lustre: lustre-OST0000: new disk, initializing [ 1578.454865] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1578.485381] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1580.379540] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing set_default_debug -1 all [ 1580.472519] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 1580.476897] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 1580.505592] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 1583.509931] Lustre: DEBUG MARKER: oleg208-client.virtnet: executing wait_import_state_mount (FULL|IDLE) failover.node=192.0.2.254@tcp0,osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1585.258227] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 1585.353356] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 1586.875062] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 1586.948350] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 1589.612082] LustreError: 122378:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 1589.614145] LustreError: 122378:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 1589.680219] Lustre: server umount lustre-OST0000 complete [ 1595.873466] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1595.873896] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1595.890117] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1595.893377] Lustre: Skipped 2 previous similar messages [ 1600.994813] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1600.997634] Lustre: Skipped 1 previous similar message [ 1600.998584] LustreError: 122578:0:(obd_class.h:479:obd_check_dev()) Device 22 not setup [ 1601.002583] LustreError: 122578:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 1601.032240] Lustre: server umount lustre-MDT0000 complete [ 1602.303422] LustreError: 119460:0:(ldlm_lockd.c:2550:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1755040967 with bad export cookie 313984267822973977 [ 1602.309877] LustreError: MGC192.168.202.108@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1612.772191] LustreError: 122782:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 1612.775211] LustreError: 122782:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 1612.871560] Lustre: server umount lustre-MDT0001 complete [ 1616.109288] Lustre: DEBUG MARKER: == conf-sanity test 90a: check max_mod_rpcs_in_flight is enforced ========================================================== 19:23:01 (1755040981) [ 1618.528186] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1618.736584] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1620.132012] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing set_default_debug -1 all [ 1623.235868] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1624.898639] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing set_default_debug -1 all [ 1626.817704] Lustre: DEBUG MARKER: oleg208-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1628.064053] Lustre: DEBUG MARKER: oleg208-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 1630.782379] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1633.203147] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing set_default_debug -1 all [ 1635.404832] Lustre: DEBUG MARKER: oleg208-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1648.419937] Lustre: *** cfs_fail_loc=159, val=0*** [ 1648.421301] Lustre: Skipped 1 previous similar message [ 1664.580373] Lustre: lustre-MDT0000: Client a2496933-e5fd-4c6c-adfc-db7047200930 (at 192.168.202.8@tcp) reconnecting [ 1665.118764] Lustre: *** cfs_fail_loc=159, val=0*** [ 1665.120787] Lustre: Skipped 4 previous similar messages [ 1681.483433] Lustre: lustre-MDT0000: Client a2496933-e5fd-4c6c-adfc-db7047200930 (at 192.168.202.8@tcp) reconnecting [ 1699.808569] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1699.811141] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1699.812085] LustreError: Skipped 1 previous similar message [ 1699.812446] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1699.815389] Lustre: Skipped 2 previous similar messages [ 1699.819459] Lustre: Skipped 1 previous similar message [ 1702.511170] LustreError: 125944:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 1702.516385] LustreError: 125944:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 1702.630451] Lustre: server umount lustre-OST0000 complete [ 1704.929484] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1704.936738] Lustre: Skipped 1 previous similar message [ 1710.049185] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1710.050937] Lustre: Skipped 3 previous similar messages [ 1710.236427] Lustre: server umount lustre-MDT0000 complete [ 1711.490216] LustreError: 123434:0:(ldlm_lockd.c:2550:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1755041076 with bad export cookie 313984267822974845 [ 1711.492034] LustreError: MGC192.168.202.108@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1711.493022] LustreError: 123434:0:(ldlm_lockd.c:2550:ldlm_cancel_handler()) Skipped 6 previous similar messages [ 1711.636745] Lustre: server umount lustre-MDT0001 complete [ 1716.612671] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing unload_modules_local [ 1717.781396] Key type lgssc unregistered [ 1717.944538] LNet: 126876:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1719.013716] LNet: Removed LNI 192.168.202.108@tcp [ 1719.431157] Key type .llcrypt unregistered [ 1719.433572] Key type ._llcrypt unregistered [ 1728.498153] Key type ._llcrypt registered [ 1728.499517] Key type .llcrypt registered [ 1728.537546] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing load_modules_local [ 1728.883136] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1728.894047] alg: No test for adler32 (adler32-zlib) [ 1729.773615] Lustre: Lustre: Build Version: 2.16.56_30_ga643f08 [ 1729.873595] LNet: Added LNI 192.168.202.108@tcp [8/256/0/180] [ 1729.875650] LNet: Accept secure, port 988 [ 1731.463317] Key type lgssc registered [ 1731.902937] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1734.850706] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1734.856347] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1736.013212] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1737.350312] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing set_default_debug -1 all [ 1740.219036] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1740.343125] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 1741.694243] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing set_default_debug -1 all [ 1743.528392] Lustre: DEBUG MARKER: oleg208-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1744.710970] Lustre: DEBUG MARKER: oleg208-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 1747.507162] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1747.715793] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1749.951208] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing set_default_debug -1 all [ 1751.832714] Lustre: DEBUG MARKER: oleg208-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1761.068844] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:42 to 0x280000401:65) [ 1767.256852] Lustre: *** cfs_fail_loc=159, val=0*** [ 1782.850099] Lustre: lustre-MDT0000: Client cf20503c-5a59-443b-9aa4-d9471190191f (at 192.168.202.8@tcp) reconnecting [ 1783.569125] Lustre: *** cfs_fail_loc=159, val=0*** [ 1798.726758] Lustre: lustre-MDT0001: Client cf20503c-5a59-443b-9aa4-d9471190191f (at 192.168.202.8@tcp) reconnecting [ 1799.256317] Lustre: *** cfs_fail_loc=159, val=0*** [ 1799.258026] Lustre: Skipped 4 previous similar messages [ 1815.620821] Lustre: lustre-MDT0001: Client cf20503c-5a59-443b-9aa4-d9471190191f (at 192.168.202.8@tcp) reconnecting [ 1816.812765] Lustre: 130942:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 1817.650583] Lustre: *** cfs_fail_loc=159, val=0*** [ 1817.652644] Lustre: Skipped 3 previous similar messages [ 1833.029859] Lustre: lustre-MDT0000: Client 1f1f312a-833b-4f97-8282-92c59e7e84b0 (at 192.168.202.8@tcp) reconnecting [ 1833.556560] Lustre: *** cfs_fail_loc=159, val=0*** [ 1833.557912] Lustre: Skipped 16 previous similar messages [ 1848.903899] Lustre: lustre-MDT0000: Client 1f1f312a-833b-4f97-8282-92c59e7e84b0 (at 192.168.202.8@tcp) reconnecting [ 1849.399669] Lustre: 131233:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 1853.409709] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1853.409938] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1853.411891] LustreError: Skipped 1 previous similar message [ 1853.412340] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1853.414893] Lustre: Skipped 1 previous similar message [ 1853.418320] Lustre: Skipped 1 previous similar message [ 1856.683125] LustreError: 131330:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 1856.748627] Lustre: server umount lustre-OST0000 complete [ 1858.528274] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1858.532488] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1863.135826] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1863.138372] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1863.143703] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1864.252603] LustreError: 131532:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup [ 1864.254408] LustreError: 131532:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 1864.278874] Lustre: server umount lustre-MDT0000 complete [ 1865.513585] LustreError: 129841:0:(ldlm_lockd.c:2550:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1755041230 with bad export cookie 8561982141082931102 [ 1865.520331] LustreError: MGC192.168.202.108@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1865.586043] LustreError: 131733:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [ 1865.588029] LustreError: 131733:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 1865.684274] Lustre: server umount lustre-MDT0001 complete [ 1870.289448] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing unload_modules_local [ 1871.452377] Key type lgssc unregistered [ 1871.594515] LNet: 132259:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1872.613280] LNet: Removed LNI 192.168.202.108@tcp [ 1872.905769] Key type .llcrypt unregistered [ 1872.906905] Key type ._llcrypt unregistered [ 1882.208907] Key type ._llcrypt registered [ 1882.210480] Key type .llcrypt registered [ 1882.253335] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing load_modules_local [ 1882.763995] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1882.781653] alg: No test for adler32 (adler32-zlib) [ 1883.651669] Lustre: Lustre: Build Version: 2.16.56_30_ga643f08 [ 1883.741488] LNet: Added LNI 192.168.202.108@tcp [8/256/0/180] [ 1883.743423] LNet: Accept secure, port 988 [ 1885.335143] Key type lgssc registered [ 1885.703459] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1888.404968] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1888.411877] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1889.573571] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1890.940700] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing set_default_debug -1 all [ 1893.915889] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1894.045252] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 1895.390179] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing set_default_debug -1 all [ 1897.211809] Lustre: DEBUG MARKER: oleg208-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1898.444275] Lustre: DEBUG MARKER: oleg208-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 1900.707285] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1900.839763] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1902.722580] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing set_default_debug -1 all [ 1903.911478] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:97) [ 1904.544730] Lustre: DEBUG MARKER: oleg208-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1914.152404] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:129) [ 1917.119541] Lustre: 135873:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 1917.738290] Lustre: 135921:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 1918.226605] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 in log params [ 1919.360131] Lustre: 136069:0:(mgs_llog.c:1345:mgs_modify_param()) MGS: modify lustre/mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 (mode = 0) failed: rc = -17 [ 1919.645344] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=7 in log params [ 1919.649297] Lustre: Skipped 1 previous similar message [ 1919.937391] Lustre: 136167:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 1924.579685] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 1924.581221] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1924.583277] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1924.586970] Lustre: Skipped 1 previous similar message [ 1924.590084] Lustre: Skipped 1 previous similar message [ 1926.891609] LustreError: 136265:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 1926.965130] Lustre: server umount lustre-OST0000 complete [ 1929.698192] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1929.699511] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1929.704437] Lustre: Skipped 1 previous similar message [ 1929.709901] Lustre: Skipped 1 previous similar message [ 1934.332533] LustreError: 136473:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup [ 1934.334508] LustreError: 136473:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 1934.363116] Lustre: server umount lustre-MDT0000 complete [ 1934.816104] LustreError: 133622:0:(ldlm_lib.c:1113:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1934.820236] LustreError: 133622:0:(ldlm_lib.c:1113:target_handle_connect()) Skipped 3 previous similar messages [ 1935.622282] LustreError: 135971:0:(ldlm_lockd.c:2550:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1755041300 with bad export cookie 8325892654126065944 [ 1935.624936] LustreError: MGC192.168.202.108@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1935.625566] LustreError: 135971:0:(ldlm_lockd.c:2550:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1935.687739] LustreError: 136674:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [ 1935.689886] LustreError: 136674:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 1935.781574] Lustre: server umount lustre-MDT0001 complete [ 1940.800854] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing unload_modules_local [ 1941.923491] Key type lgssc unregistered [ 1942.070446] LNet: 137200:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1943.141443] LNet: Removed LNI 192.168.202.108@tcp [ 1943.444948] Key type .llcrypt unregistered [ 1943.446241] Key type ._llcrypt unregistered [ 1952.520101] Key type ._llcrypt registered [ 1952.521057] Key type .llcrypt registered [ 1952.566653] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing load_modules_local [ 1953.058926] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1953.073746] alg: No test for adler32 (adler32-zlib) [ 1953.935391] Lustre: Lustre: Build Version: 2.16.56_30_ga643f08 [ 1954.026743] LNet: Added LNI 192.168.202.108@tcp [8/256/0/180] [ 1954.028404] LNet: Accept secure, port 988 [ 1955.615141] Key type lgssc registered [ 1956.059558] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1959.177428] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1959.186184] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1960.370208] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1961.673099] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing set_default_debug -1 all [ 1965.098678] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1965.276136] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 1966.694995] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing set_default_debug -1 all [ 1968.774802] Lustre: DEBUG MARKER: oleg208-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1970.072929] Lustre: DEBUG MARKER: oleg208-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 1972.836192] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1973.002630] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1975.083234] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:129) [ 1975.195914] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing set_default_debug -1 all [ 1977.241162] Lustre: DEBUG MARKER: oleg208-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1985.326371] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:161) [ 1989.870602] Lustre: *** cfs_fail_loc=159, val=0*** [ 2006.090834] Lustre: lustre-MDT0000: Client 29c18ff6-97c5-4e9f-830d-d326e48bb57f (at 192.168.202.8@tcp) reconnecting [ 2011.106774] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2011.106940] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2011.111967] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2011.113039] LustreError: Skipped 1 previous similar message [ 2011.116348] Lustre: Skipped 1 previous similar message [ 2013.679172] LustreError: 140947:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 2013.769136] Lustre: server umount lustre-OST0000 complete [ 2016.225643] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2016.231188] Lustre: Skipped 1 previous similar message [ 2016.233602] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2016.235891] Lustre: Skipped 1 previous similar message [ 2021.256129] LustreError: 141149:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup [ 2021.259584] LustreError: 141149:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 2021.288836] Lustre: server umount lustre-MDT0000 complete [ 2021.344062] LustreError: 138563:0:(ldlm_lib.c:1113:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2021.347879] LustreError: 138563:0:(ldlm_lib.c:1113:target_handle_connect()) Skipped 3 previous similar messages [ 2022.509205] LustreError: 138543:0:(ldlm_lockd.c:2550:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1755041387 with bad export cookie 3368834186310254864 [ 2022.512259] LustreError: 138543:0:(ldlm_lockd.c:2550:ldlm_cancel_handler()) Skipped 1 previous similar message [ 2022.512526] LustreError: MGC192.168.202.108@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2022.572794] LustreError: 141350:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [ 2022.574463] LustreError: 141350:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 2022.662929] Lustre: server umount lustre-MDT0001 complete [ 2027.601593] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing unload_modules_local [ 2028.655336] Key type lgssc unregistered [ 2028.812581] LNet: 141877:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2029.861678] LNet: Removed LNI 192.168.202.108@tcp [ 2030.332620] Key type .llcrypt unregistered [ 2030.334972] Key type ._llcrypt unregistered [ 2039.526595] Key type ._llcrypt registered [ 2039.527479] Key type .llcrypt registered [ 2039.568946] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing load_modules_local [ 2040.102346] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2040.111580] alg: No test for adler32 (adler32-zlib) [ 2040.983285] Lustre: Lustre: Build Version: 2.16.56_30_ga643f08 [ 2041.070263] LNet: Added LNI 192.168.202.108@tcp [8/256/0/180] [ 2041.071798] LNet: Accept secure, port 988 [ 2042.655195] Key type lgssc registered [ 2043.048897] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2045.841792] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 2045.850668] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2047.012191] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2048.499517] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing set_default_debug -1 all [ 2051.554771] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2051.680819] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2053.204775] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing set_default_debug -1 all [ 2055.213102] Lustre: DEBUG MARKER: oleg208-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2056.611603] Lustre: DEBUG MARKER: oleg208-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2059.134746] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2059.266323] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2061.251791] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing set_default_debug -1 all [ 2061.351111] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:161) [ 2063.400289] Lustre: DEBUG MARKER: oleg208-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2073.647820] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:170 to 0x280000401:193) [ 2087.612698] Lustre: lustre-OST0000: evicting 1c002111-78d0-4fb4-93a5-13a622e0f355 (at 192.168.202.8@tcp) by administrative request [ 2087.615692] Lustre: Skipped 1 previous similar message [ 2112.480571] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2112.483672] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2112.489233] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2114.530921] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2114.531261] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2114.534208] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2117.803243] LustreError: 146025:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 2117.867019] Lustre: server umount lustre-OST0000 complete [ 2119.649867] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2119.650798] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2119.654668] Lustre: Skipped 1 previous similar message [ 2119.657167] Lustre: Skipped 2 previous similar messages [ 2124.769086] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2124.771245] Lustre: Skipped 1 previous similar message [ 2125.374745] LustreError: 146227:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup [ 2125.378136] LustreError: 146227:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 2125.406294] Lustre: server umount lustre-MDT0000 complete [ 2126.665145] LustreError: 143957:0:(ldlm_lockd.c:2550:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1755041491 with bad export cookie 17261607801389264016 [ 2126.668485] LustreError: 143957:0:(ldlm_lockd.c:2550:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2126.668944] LustreError: MGC192.168.202.108@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2126.733840] LustreError: 146430:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [ 2126.735670] LustreError: 146430:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 2126.827462] Lustre: server umount lustre-MDT0001 complete [ 2131.754633] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing unload_modules_local [ 2133.020871] Key type lgssc unregistered [ 2133.162541] LNet: 146956:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2134.181424] LNet: Removed LNI 192.168.202.108@tcp [ 2134.499730] Key type .llcrypt unregistered [ 2134.501712] Key type ._llcrypt unregistered [ 2158.710112] Key type ._llcrypt registered [ 2158.711695] Key type .llcrypt registered [ 2158.750641] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing load_modules_local [ 2159.071875] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2159.081983] alg: No test for adler32 (adler32-zlib) [ 2159.947231] Lustre: Lustre: Build Version: 2.16.56_30_ga643f08 [ 2160.038232] LNet: Added LNI 192.168.202.108@tcp [8/256/0/180] [ 2160.039953] LNet: Accept secure, port 988 [ 2161.631190] Key type lgssc registered [ 2162.007489] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2164.770190] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 2164.776302] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2165.945989] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2167.273373] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing set_default_debug -1 all [ 2170.203664] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2170.352041] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2171.680526] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing set_default_debug -1 all [ 2173.442119] Lustre: DEBUG MARKER: oleg208-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2174.586799] Lustre: DEBUG MARKER: oleg208-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2176.806359] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2176.935378] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2178.814858] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing set_default_debug -1 all [ 2180.011810] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:193) [ 2180.619183] Lustre: DEBUG MARKER: oleg208-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2191.276106] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:195 to 0x280000401:225) [ 2200.544070] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2200.547045] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2200.550989] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2201.571942] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2201.573566] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2201.573598] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2204.779299] LustreError: 151644:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 2204.852646] Lustre: server umount lustre-OST0000 complete [ 2206.175970] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2206.180140] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2206.186972] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2206.189572] Lustre: Skipped 1 previous similar message [ 2211.809188] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2211.811316] Lustre: Skipped 2 previous similar messages [ 2212.220045] LustreError: 151847:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup [ 2212.223346] LustreError: 151847:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 2212.250957] Lustre: server umount lustre-MDT0000 complete [ 2213.531432] LustreError: 149366:0:(ldlm_lockd.c:2550:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1755041578 with bad export cookie 16080261297646363527 [ 2213.536482] LustreError: MGC192.168.202.108@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2213.604955] LustreError: 152049:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [ 2213.608304] LustreError: 152049:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 2213.708459] Lustre: server umount lustre-MDT0001 complete [ 2218.376387] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing unload_modules_local [ 2219.544270] Key type lgssc unregistered [ 2219.676530] LNet: 152575:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2220.709680] LNet: Removed LNI 192.168.202.108@tcp [ 2221.030726] Key type .llcrypt unregistered [ 2221.032095] Key type ._llcrypt unregistered [ 2230.804898] Key type ._llcrypt registered [ 2230.806324] Key type .llcrypt registered [ 2230.856417] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing load_modules_local [ 2231.459528] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2231.466986] alg: No test for adler32 (adler32-zlib) [ 2232.328498] Lustre: Lustre: Build Version: 2.16.56_30_ga643f08 [ 2232.415070] LNet: Added LNI 192.168.202.108@tcp [8/256/0/180] [ 2232.416418] LNet: Accept secure, port 988 [ 2233.999132] Key type lgssc registered [ 2234.419515] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2237.121575] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2239.519885] Lustre: DEBUG MARKER: == conf-sanity test 100: check lshowmount lists MGS, MDT, OST and 0@lo ========================================================== 19:33:25 (1755041605) [ 2244.979204] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing set_hostid [ 2248.410890] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing load_modules_local [ 2251.974105] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2254.459776] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2257.915070] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2262.245034] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2266.732792] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2266.761455] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 2266.770344] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2267.980440] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2268.019667] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2268.137816] Lustre: lustre-MDT0000: new disk, initializing [ 2268.239714] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2268.264946] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2270.997529] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing set_default_debug -1 all [ 2278.066619] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2278.119605] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2278.184164] Lustre: 156747:0:(mgs_llog.c:1345:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2278.217803] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 2278.220988] Lustre: Skipped 1 previous similar message [ 2278.278494] Lustre: lustre-MDT0001: new disk, initializing [ 2278.340482] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2278.366360] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 2278.372901] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2280.790632] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing set_default_debug -1 all [ 2285.562274] Lustre: DEBUG MARKER: oleg208-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2287.702410] Lustre: DEBUG MARKER: oleg208-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2291.863918] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2291.914511] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2292.154600] Lustre: lustre-OST0000: new disk, initializing [ 2292.161846] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 2292.231912] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2295.873676] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing set_default_debug -1 all [ 2301.583600] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 2301.593161] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2301.714215] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2304.505868] Lustre: DEBUG MARKER: oleg208-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2308.392370] hrtimer: interrupt took 2508798 ns [ 2322.402159] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2322.410583] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2322.462017] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2322.475529] Lustre: Skipped 1 previous similar message [ 2325.175861] LustreError: 159310:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 2325.984848] Lustre: server umount lustre-OST0000 complete [ 2334.688402] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2334.693318] LustreError: Skipped 1 previous similar message [ 2334.695480] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2334.712900] Lustre: Skipped 1 previous similar message [ 2334.727484] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2337.760635] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2337.763783] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2339.517228] LustreError: 159513:0:(obd_class.h:479:obd_check_dev()) Device 22 not setup [ 2339.532621] LustreError: 159513:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 2339.723215] Lustre: server umount lustre-MDT0000 complete [ 2346.665905] LustreError: 156738:0:(ldlm_lockd.c:2550:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1755041711 with bad export cookie 17878233760342103777 [ 2346.688193] LustreError: MGC192.168.202.108@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2347.008813] LustreError: 159715:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 2347.011282] LustreError: 159715:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 2347.732202] Lustre: server umount lustre-MDT0001 complete [ 2368.843896] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing unload_modules_local [ 2373.122957] Key type lgssc unregistered [ 2373.507749] LNet: 160264:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2374.582093] LNet: Removed LNI 192.168.202.108@tcp [ 2375.909943] Key type .llcrypt unregistered [ 2375.915858] Key type ._llcrypt unregistered [ 2399.067403] Key type ._llcrypt registered [ 2399.068766] Key type .llcrypt registered [ 2399.202564] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing load_modules_local [ 2399.906689] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2400.071650] alg: No test for adler32 (adler32-zlib) [ 2401.234304] Lustre: Lustre: Build Version: 2.16.56_30_ga643f08 [ 2401.553828] LNet: Added LNI 192.168.202.108@tcp [8/256/0/180] [ 2401.582855] LNet: Accept secure, port 988 [ 2403.319254] Key type lgssc registered [ 2404.538843] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2411.921338] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 2411.942227] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2413.331216] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2416.654822] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing set_default_debug -1 all [ 2423.777897] LustreError: 161646:0:(ldlm_lib.c:1113:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2423.805468] LustreError: 161646:0:(ldlm_lib.c:1113:target_handle_connect()) Skipped 2 previous similar messages [ 2425.173694] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2425.434978] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2428.780177] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing set_default_debug -1 all [ 2433.144052] Lustre: DEBUG MARKER: oleg208-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2436.137944] Lustre: DEBUG MARKER: oleg208-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2441.829269] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2442.233710] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2447.140692] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing set_default_debug -1 all [ 2452.001769] Lustre: DEBUG MARKER: oleg208-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2458.632291] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:3 to 0x280000401:33) [ 2468.280944] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 2468.313197] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2468.322837] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 2468.340471] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2468.365361] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 192.168.202.108@tcp (at 0@lo) [ 2469.598666] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 2469.610252] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2469.623945] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 2469.629738] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2469.648674] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 192.168.202.108@tcp (at 0@lo) [ 2470.714798] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 2470.748833] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2470.763921] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 2470.767428] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2470.775072] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 192.168.202.108@tcp (at 0@lo) [ 2471.649080] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2472.589341] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2473.533407] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 2473.540687] Lustre: Skipped 2 previous similar messages [ 2473.566692] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2473.579231] Lustre: Skipped 2 previous similar messages [ 2473.589992] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 2473.596420] Lustre: Skipped 2 previous similar messages [ 2473.605875] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2473.622691] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 192.168.202.108@tcp (at 0@lo) [ 2473.625730] Lustre: Skipped 2 previous similar messages [ 2474.642927] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2475.600522] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2476.572442] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2477.609642] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 2477.613329] Lustre: Skipped 3 previous similar messages [ 2477.636867] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2477.647505] Lustre: Skipped 3 previous similar messages [ 2477.650338] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 2477.653913] Lustre: Skipped 3 previous similar messages [ 2477.655568] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2477.662613] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 192.168.202.108@tcp (at 0@lo) [ 2477.682655] Lustre: Skipped 3 previous similar messages [ 2478.745100] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2479.622528] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2480.560455] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2481.580555] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2482.578714] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2483.437502] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2484.559345] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2485.445939] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2486.506336] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 2486.509804] Lustre: Skipped 8 previous similar messages [ 2486.522419] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2486.536300] Lustre: Skipped 8 previous similar messages [ 2486.540696] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 2486.545641] Lustre: Skipped 8 previous similar messages [ 2486.547957] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2486.568979] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 192.168.202.108@tcp (at 0@lo) [ 2486.576089] Lustre: Skipped 8 previous similar messages [ 2487.381876] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2488.351873] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2489.185851] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2490.160794] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2491.267434] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2492.208708] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2493.144852] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2494.049910] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2494.774295] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2495.584296] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2496.570293] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2497.578375] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2498.500669] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2499.320581] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2500.247889] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2501.055686] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2501.996987] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2502.839139] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 2502.845589] Lustre: Skipped 17 previous similar messages [ 2502.854783] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2502.865577] Lustre: Skipped 17 previous similar messages [ 2502.868609] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 2502.872309] Lustre: Skipped 17 previous similar messages [ 2502.876537] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2502.885106] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 192.168.202.108@tcp (at 0@lo) [ 2502.891243] Lustre: Skipped 17 previous similar messages [ 2503.787215] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2504.590598] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2505.511817] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2506.390586] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2507.207691] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2508.048856] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2508.838684] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2509.577341] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2510.371482] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2511.268590] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2512.073377] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2512.740762] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2513.561709] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2514.461039] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2515.331754] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2516.156850] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2517.020160] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2517.896857] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2518.889824] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2519.736798] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2520.826700] LustreError: 161666:0:(osp_precreate.c:654:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 2520.839339] LustreError: 161666:0:(osp_precreate.c:1358:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 2520.851727] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2521.709935] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2522.603639] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2523.484959] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2524.331401] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2525.164662] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2525.988450] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2526.755824] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2527.557408] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2528.257830] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2529.079235] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2529.790803] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2530.504491] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2531.342369] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2532.300702] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2533.174361] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2534.013870] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2534.859592] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 2534.863303] Lustre: Skipped 37 previous similar messages [ 2534.884795] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2534.893344] Lustre: Skipped 37 previous similar messages [ 2534.898395] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 2534.902911] Lustre: Skipped 37 previous similar messages [ 2534.908777] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2534.917098] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 192.168.202.108@tcp (at 0@lo) [ 2534.932153] Lustre: Skipped 37 previous similar messages [ 2535.659296] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2536.492573] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2537.305152] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2538.261669] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2539.144696] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2540.057745] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2540.898152] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2541.766978] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2542.526861] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2543.420873] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2544.084239] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2544.872754] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2545.641334] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2546.316770] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2547.044888] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2547.840953] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2548.523737] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2549.262113] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2550.143796] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2550.953955] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2551.877803] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2552.806899] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2553.717311] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2554.790324] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2555.747452] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2556.910256] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2557.914398] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2559.007102] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2560.073113] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2560.995289] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2561.900196] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2562.764968] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2563.649308] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2564.569587] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2565.507420] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2566.530279] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2567.416578] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2568.330532] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2569.130371] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2570.053297] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2571.107328] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2571.974908] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2572.776022] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2573.539061] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2574.458304] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2575.337108] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2576.276900] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2577.333744] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2578.289378] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2579.290704] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2580.188722] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2581.076891] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2582.125191] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2583.106228] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2584.043894] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2585.014240] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2585.937972] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2586.761703] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2587.500179] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2588.315860] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2589.101081] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2590.297081] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2591.197415] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2592.117577] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2592.988815] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2594.001590] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2595.075799] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2595.969529] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2596.879578] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2597.933734] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2598.958464] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 2598.961928] Lustre: Skipped 70 previous similar messages [ 2598.992530] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2599.004787] Lustre: Skipped 70 previous similar messages [ 2599.010644] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 2599.015094] Lustre: Skipped 70 previous similar messages [ 2599.018725] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2599.028496] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 192.168.202.108@tcp (at 0@lo) [ 2599.031841] Lustre: Skipped 70 previous similar messages [ 2599.991049] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2600.918858] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2601.836366] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2602.779546] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2603.913145] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2604.837075] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2605.874599] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2606.721992] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2607.466272] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2608.360653] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2609.228575] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2610.280152] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2611.118677] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2611.982031] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2613.088045] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2614.072404] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2614.961590] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2615.988402] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2616.806546] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2617.836254] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2618.732072] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2619.620215] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2620.893728] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2622.147903] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2623.412710] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2624.447843] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2625.229613] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2626.186124] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2627.481131] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2628.612574] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2629.549634] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2630.726844] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2632.106610] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2633.367134] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2634.799171] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2636.442720] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2637.533158] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2638.579505] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2640.020567] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2641.432428] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2642.515457] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2644.050378] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2645.232046] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2646.423909] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2647.858624] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2649.462817] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2650.775975] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2652.133732] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2653.489270] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2654.763800] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2656.117350] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2657.261404] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2658.338309] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2659.369661] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2660.438991] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2661.731708] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2662.931648] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2663.876301] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2664.918176] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2666.087254] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2667.206063] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2667.968894] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2668.711251] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2669.635894] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2670.655796] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2671.667819] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2672.890260] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2674.740806] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2676.253767] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2677.832948] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2679.462163] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2680.827855] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2682.024726] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2683.304499] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2684.704191] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2686.236696] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2687.559422] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2688.887572] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2690.220429] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2691.424977] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2692.750101] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2694.251217] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2695.462268] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2696.828417] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2698.038892] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2699.564053] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2701.068129] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2702.367293] LustreError: 161666:0:(osp_precreate.c:654:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 2702.376716] LustreError: 161666:0:(osp_precreate.c:1358:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 2702.400794] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2703.630665] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2704.943483] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2706.137827] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2707.503074] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2708.712091] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2709.833910] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2711.116438] LustreError: 161666:0:(osp_precreate.c:654:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 2711.132983] LustreError: 161666:0:(osp_precreate.c:1358:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 2711.136466] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2712.037384] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2713.419054] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2714.566308] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2715.662341] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2717.226349] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2718.337581] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2719.471633] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2720.813747] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2721.948411] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2722.759453] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2723.780122] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2724.630429] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2725.605759] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2726.400020] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2727.476696] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 2727.479320] Lustre: Skipped 109 previous similar messages [ 2727.516343] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2727.524758] Lustre: Skipped 109 previous similar messages [ 2727.539835] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 2727.549098] Lustre: Skipped 109 previous similar messages [ 2727.555955] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2727.564482] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 192.168.202.108@tcp (at 0@lo) [ 2727.590938] Lustre: Skipped 109 previous similar messages [ 2728.673837] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2729.596947] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2730.466290] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2731.617081] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2732.623573] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2733.574083] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2734.508148] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2735.430079] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2736.340783] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2737.364503] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2738.198957] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2739.143210] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2740.064810] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2741.032203] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2742.091711] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2743.106378] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2744.106738] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2745.265138] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2746.599944] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2747.680792] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2748.637715] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2750.406128] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2751.207109] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2752.668050] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2753.869516] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2755.066221] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2756.287862] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2757.302156] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2758.543464] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2759.659899] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2760.607840] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2761.750855] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2762.626343] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2763.909986] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2765.023790] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2766.071880] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2767.366692] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2768.650820] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2769.742664] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2771.036141] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2772.177416] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2773.343186] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2774.365157] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2775.366820] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2776.247225] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2777.104602] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2777.983895] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2779.218342] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2780.125092] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2780.937283] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2781.861705] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2782.794944] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2783.483362] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2784.189811] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2785.064727] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2785.947497] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2786.773957] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2787.554417] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2788.531777] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2789.924968] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2790.717848] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2791.869319] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2793.019097] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2794.010445] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2795.171904] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2796.431532] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2797.606675] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2798.753554] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2799.645465] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2800.771506] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2802.216254] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2803.258611] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2804.688434] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2806.093887] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2807.400372] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2809.190702] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2810.649082] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2812.290714] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2813.647678] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2814.965760] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2816.680164] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2818.210966] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2819.522962] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2820.797515] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2821.949725] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2823.365971] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2824.725477] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2825.940311] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2827.179561] LustreError: 161666:0:(osp_precreate.c:654:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 2827.200634] LustreError: 161666:0:(osp_precreate.c:1358:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 2827.209634] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2828.576416] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2829.673992] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2830.689588] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2831.735479] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2832.540281] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2833.489658] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2834.274371] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2835.109307] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2835.959607] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2836.804468] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2837.497785] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2838.358853] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2839.107788] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2839.776989] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2840.513846] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2841.201697] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2841.875650] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2842.808755] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2843.549545] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2844.197844] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2845.004682] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2845.680217] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2846.280524] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2846.868958] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2847.626527] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2848.439924] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2849.260118] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2850.059887] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2850.731766] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2851.497923] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2852.233497] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2852.973728] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2853.785490] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2854.407615] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2855.112239] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2856.005826] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2856.758345] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2857.580921] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2858.324837] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2859.086662] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2859.919602] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2860.693351] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2861.365677] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2862.167540] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2862.810952] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2863.562446] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2864.283946] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2865.046046] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2865.708886] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2866.389518] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2867.279050] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2868.011984] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2868.813758] LustreError: 161666:0:(osp_precreate.c:654:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 2868.817921] LustreError: 161666:0:(osp_precreate.c:1358:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 2868.822265] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2869.529550] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2870.221463] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2871.092460] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2871.854321] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2872.610335] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2873.328437] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2874.025834] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2874.757087] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2875.580290] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2876.371373] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2877.179602] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2877.859768] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2878.628637] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2879.357814] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2880.085769] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2880.776581] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2881.531962] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2882.228631] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2882.869710] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2883.579223] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2884.149793] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2884.853942] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2885.474802] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2886.149978] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2886.718723] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2887.331991] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2888.007795] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2888.622602] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2889.186268] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2889.715236] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2890.210882] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2890.787579] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2891.338379] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2891.985554] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2892.555441] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2893.217858] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2893.838051] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2894.476051] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2895.062872] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2895.772673] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2896.426942] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2897.228593] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2898.009643] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2898.875086] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2899.632243] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2900.368699] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2901.101976] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2901.891503] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2902.501228] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2903.129192] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2903.733272] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2904.278198] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2904.924158] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2905.518164] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2906.107161] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2906.716295] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2907.308850] LustreError: 161666:0:(osp_precreate.c:654:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 2907.314349] LustreError: 161666:0:(osp_precreate.c:1358:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 2907.318326] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2907.905927] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2908.551352] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2909.094431] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2909.742686] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2910.353452] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2910.886973] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2911.405428] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2911.870940] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2912.369560] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2912.804825] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2913.282974] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2913.810298] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2914.369480] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2915.050566] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2915.687223] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2916.400150] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2916.940808] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2917.497832] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2918.044705] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2918.591693] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2919.265390] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2919.866342] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2920.451910] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2920.951908] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2921.510337] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2922.094505] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2922.622925] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2923.265230] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2923.900813] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2924.560317] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2925.239346] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2925.889319] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2926.486250] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2927.008631] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2927.602980] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2928.198033] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2928.759646] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2929.295704] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2929.888158] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2930.499864] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2931.028975] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2931.537950] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2932.090760] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2932.824666] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2933.424640] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2934.125551] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2934.707194] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2935.350159] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2936.106152] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2936.874195] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2937.580269] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2938.248116] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2938.921792] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2939.606785] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2940.307497] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2941.061310] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2941.885369] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2942.642857] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2943.256803] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2943.854106] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2944.410985] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2945.069970] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2945.683829] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2946.270252] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2946.884389] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2947.595787] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2948.195545] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2948.889343] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2949.524719] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2950.147535] LustreError: 161666:0:(osp_precreate.c:654:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 2950.153497] LustreError: 161666:0:(osp_precreate.c:1358:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 2950.157858] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2950.730261] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2951.293897] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2951.997151] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2952.675975] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2953.324981] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2953.965155] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2954.620468] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2955.216772] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2955.806792] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2956.420617] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2957.076482] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2957.677445] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2958.282411] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2958.836577] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2959.388881] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2960.253496] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2961.011475] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2961.711312] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2962.547653] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2963.280936] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2963.991985] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2964.683672] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2965.292343] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2965.912272] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2966.433572] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2967.034870] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2967.659519] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2968.360388] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2969.033367] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2969.660416] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2970.258972] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2970.899056] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2971.594738] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2972.174475] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2972.881446] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2973.553672] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2974.166670] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2974.845336] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2975.526485] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2976.164530] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2976.718625] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2977.238735] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2977.890980] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2978.478229] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2979.093967] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2979.737168] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2980.408122] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2981.039291] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2981.702520] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2982.401296] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2983.087242] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2983.754492] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 2983.763983] Lustre: Skipped 320 previous similar messages [ 2983.777527] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2983.787261] Lustre: Skipped 320 previous similar messages [ 2983.791317] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 2983.793967] Lustre: Skipped 320 previous similar messages [ 2983.796333] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2983.804159] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 192.168.202.108@tcp (at 0@lo) [ 2983.812232] Lustre: Skipped 320 previous similar messages [ 2984.500995] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2985.140578] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2985.712689] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2986.294704] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2986.853616] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2987.488629] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2988.125768] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2988.708388] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2989.339698] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2989.941479] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2990.673618] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2991.232073] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2991.809444] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2992.551264] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2993.313845] LustreError: 161666:0:(osp_precreate.c:654:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 2993.326034] LustreError: 161666:0:(osp_precreate.c:1358:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 2993.328674] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2993.921270] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2994.541829] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2995.157758] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2995.878556] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2996.631430] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2997.400320] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2998.128612] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2998.835423] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2999.539821] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3000.228657] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3000.811409] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3001.462656] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3001.945794] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3002.430064] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3003.010051] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3003.496213] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3003.972801] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3004.594912] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3005.179662] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3005.732886] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3006.240171] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3006.819240] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3007.447500] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3008.000440] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3008.682989] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3009.494539] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3010.173989] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3010.864137] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3011.553674] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3012.243415] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3012.750328] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3013.302523] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3013.897808] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3014.452275] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3015.060201] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3015.766189] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3016.397718] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3016.963374] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3017.501549] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3018.053764] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3018.609360] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3019.207536] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3019.764174] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3020.405964] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3021.000189] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3021.596557] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3022.184535] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3022.888462] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3023.556360] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3024.111032] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3024.831428] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3025.489608] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3026.336595] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3027.155247] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3028.092515] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3028.833897] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3029.550554] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3030.347444] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3031.206535] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3032.095551] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3032.880879] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3033.647798] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3034.365632] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3035.137567] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3035.894434] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3036.579294] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3037.303395] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3038.081955] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3038.756205] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3039.434075] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3040.002331] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3040.532153] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3041.096375] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3041.742716] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3042.407344] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3043.112206] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3043.745036] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3044.395106] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3045.067643] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3045.749711] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3046.530508] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3047.110914] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3047.862740] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3048.492431] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3049.093428] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3049.698547] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3050.327436] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3050.944606] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3051.726128] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3052.479454] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3053.161297] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3053.822967] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3054.524261] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3055.126345] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3055.815440] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3056.410559] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3056.972694] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3057.716277] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3058.364505] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3058.977718] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3059.638358] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3060.242117] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3061.027335] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3061.768183] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3062.553795] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3063.257400] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3063.977326] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3064.779336] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3065.579884] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3066.444452] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3067.332417] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3068.010532] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3068.844412] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3069.806323] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3070.991904] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3071.875340] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3073.002949] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3073.946926] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3075.090895] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3076.100419] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3077.060995] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3077.962360] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3079.276546] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3080.429431] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3081.341502] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3082.258704] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3083.102791] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3083.896422] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3084.580959] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3085.206301] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3085.899520] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3086.516670] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3087.155156] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3087.825583] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3088.449550] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3089.061807] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3089.591810] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3090.049039] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3090.623092] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3091.169745] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3091.779668] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3092.374809] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3093.050850] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3093.601875] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3094.203827] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3094.685212] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3095.128120] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3095.570450] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3096.020425] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3096.448184] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3096.867079] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3097.282794] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3097.697068] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3098.142169] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3098.583609] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3099.036647] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3099.555623] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3100.072409] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3100.541202] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3100.962284] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3101.367811] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3101.814480] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3102.277170] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3102.772626] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3103.219861] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3103.704296] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3104.195135] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3104.679409] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3105.157580] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3105.665988] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3106.233702] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3106.731227] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3107.284131] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3107.738387] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3108.166855] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3108.661729] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3109.098274] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3109.505739] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3110.030875] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3110.489929] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3110.964985] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3111.400077] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3111.909740] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3112.365221] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3112.786680] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3113.204495] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3113.670251] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3114.121314] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3114.555783] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3114.935517] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3115.298380] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3115.732913] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3116.210334] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3116.644151] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3117.054652] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3117.526350] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3118.030322] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3118.471731] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3118.957785] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3119.357844] LustreError: 161666:0:(osp_precreate.c:654:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 3119.362394] LustreError: 161666:0:(osp_precreate.c:1358:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 3119.364393] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3119.818447] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3120.350219] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3120.860644] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3121.409143] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3122.000746] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3122.499960] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3123.047489] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3123.639428] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3124.204269] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3124.678781] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3125.156137] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3125.565625] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3126.036024] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3126.544605] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3127.077192] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3127.477084] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3127.855830] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3128.275865] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3128.664675] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3129.053885] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3129.428614] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3129.839385] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3130.236393] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3130.630227] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3131.021073] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3131.374191] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3131.759278] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3132.154072] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3132.543122] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3132.890423] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3133.237957] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3133.610535] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3134.052174] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3134.455978] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3134.880577] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3135.322047] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3135.713413] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3136.111657] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3136.585401] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3137.119398] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3137.606505] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3138.059284] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3138.564243] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3139.063733] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3139.583389] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3140.043725] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3140.532110] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3140.965904] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3141.424568] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3141.877576] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3142.311505] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3142.736304] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3143.152606] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3143.610784] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3144.044473] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3144.495696] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3144.866494] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3145.249981] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3145.614968] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3145.965358] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3146.302648] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3146.653359] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3147.014931] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3147.416857] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3147.772072] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3148.152365] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3148.531636] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3148.933697] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3149.386226] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3149.784690] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3150.252164] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3150.643651] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3151.030429] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3151.450822] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3151.828232] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3152.249987] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3152.709061] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3153.084669] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3153.467273] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3153.808443] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3154.147356] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3154.502407] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3154.891534] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3155.294972] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3155.673821] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3156.056152] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3156.454086] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3156.873754] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3157.302734] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3157.710672] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3158.112983] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3158.490397] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3158.845112] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3159.208630] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3159.610629] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3159.991097] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3160.369343] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3160.747350] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3161.131570] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3161.485354] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3161.887627] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3162.317531] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3162.711887] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3163.190590] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3163.648978] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3164.056590] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3164.479034] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3164.904102] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3165.296680] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3165.736931] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3166.177414] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3166.625786] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3167.063277] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3167.476688] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3167.853754] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3168.271574] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3168.663686] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3169.055443] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3169.444752] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3169.836032] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3170.243339] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3170.623363] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3170.994284] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3171.350479] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3171.714479] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3172.108809] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3172.513654] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3172.918874] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3173.285952] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3173.689181] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3174.128406] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3174.507615] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3174.938876] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3175.349518] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3175.778507] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3176.203498] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3176.709645] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3177.102587] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3177.544519] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3177.969964] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3178.372859] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3178.771977] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3179.135859] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3179.536682] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3179.986414] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3180.455394] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3180.910910] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3181.338373] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3181.778859] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3182.198121] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3182.624286] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3183.095561] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3183.565841] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3184.038825] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3184.595762] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3185.014579] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3185.457119] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3185.853663] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3186.263496] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3186.684277] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3187.118980] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3187.559765] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3187.946240] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3188.349608] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3188.726530] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3189.094570] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3189.452846] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3189.789980] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3190.129227] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3190.466443] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3190.787213] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3191.114232] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3191.439779] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3191.788919] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3192.134902] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3192.480229] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3192.849900] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3193.238365] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3193.609884] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3193.984814] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3194.360427] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3194.736484] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3195.133661] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3195.598895] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3196.033381] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3196.453564] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3196.856374] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3197.250131] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3197.816310] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3198.273612] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3198.702277] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3199.109931] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3199.532474] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3199.990929] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3200.447880] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3200.883190] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3201.290596] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3201.761291] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3202.189446] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3202.553969] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3202.942082] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3203.332031] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3203.698635] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3204.087745] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3204.562053] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3205.005873] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3205.415562] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3205.827943] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3501.360739] LustreError: 163232:0:(ofd_dev.c:1776:ofd_create_hdl()) lustre-OST0000: unable to precreate: rc = -28 [ 3501.366269] LustreError: 161666:0:(osp_precreate.c:654:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -28 [ 3710.435591] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3710.437749] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3710.441408] Lustre: Skipped 424 previous similar messages [ 3710.445262] Lustre: Skipped 1 previous similar message [ 3712.049974] LustreError: 215505:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 3712.167374] Lustre: server umount lustre-OST0000 complete [ 3715.552872] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3715.555161] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3715.558815] Lustre: Skipped 1 previous similar message [ 3719.814045] LustreError: 215705:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup [ 3719.817060] LustreError: 215705:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 3719.927728] Lustre: server umount lustre-MDT0000 complete [ 3720.672635] LustreError: 215109:0:(ldlm_lib.c:1113:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3720.677694] LustreError: 215109:0:(ldlm_lib.c:1113:target_handle_connect()) Skipped 1 previous similar message [ 3721.318544] LustreError: 161627:0:(ldlm_lockd.c:2550:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1755043086 with bad export cookie 12908842987526184379 [ 3721.322397] LustreError: MGC192.168.202.108@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3721.325508] LustreError: 161627:0:(ldlm_lockd.c:2550:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 3721.390127] LustreError: 215906:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [ 3721.393114] LustreError: 215906:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 3721.474722] Lustre: server umount lustre-MDT0001 complete [ 3726.551530] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing unload_modules_local [ 3727.698094] Key type lgssc unregistered [ 3727.833481] LNet: 216433:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3728.869392] LNet: Removed LNI 192.168.202.108@tcp [ 3729.199509] Key type .llcrypt unregistered [ 3729.200947] Key type ._llcrypt unregistered [ 3739.199631] Key type ._llcrypt registered [ 3739.200885] Key type .llcrypt registered [ 3739.245461] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing load_modules_local [ 3739.692028] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3739.705810] alg: No test for adler32 (adler32-zlib) [ 3740.581461] Lustre: Lustre: Build Version: 2.16.56_30_ga643f08 [ 3740.674960] LNet: Added LNI 192.168.202.108@tcp [8/256/0/180] [ 3740.676467] LNet: Accept secure, port 988 [ 3742.255205] Key type lgssc registered [ 3742.673677] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3745.569274] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3745.579263] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3746.748726] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3748.131044] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing set_default_debug -1 all [ 3751.225863] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3751.380582] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3752.847797] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing set_default_debug -1 all [ 3754.886221] Lustre: DEBUG MARKER: oleg208-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 3756.279550] Lustre: DEBUG MARKER: oleg208-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 3758.722689] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3758.846209] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3760.863624] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing set_default_debug -1 all [ 3762.914688] Lustre: DEBUG MARKER: oleg208-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3771.176095] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49666 to 0x280000401:49697) [ 3775.465116] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 3776.089258] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3776.095190] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 3776.098950] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3776.103637] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 192.168.202.108@tcp (at 0@lo) [ 3776.103685] LustreError: 217815:0:(osp_precreate.c:1266:osp_precreate_thread()) cfs_fail_timeout id 2107 sleeping for 20000ms [ 3778.628359] Lustre: lustre-OST0000: Not available for connect from 192.168.202.8@tcp (stopping) [ 3781.601706] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3781.602399] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3781.606245] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3781.609516] Lustre: Skipped 1 previous similar message [ 3783.737761] Lustre: lustre-OST0000: Not available for connect from 192.168.202.8@tcp (stopping) [ 3786.722144] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3786.724901] Lustre: Skipped 1 previous similar message [ 3791.327126] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 3791.345022] LustreError: 220193:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 3791.412475] Lustre: server umount lustre-OST0000 complete [ 3796.183055] LustreError: 217815:0:(osp_precreate.c:1266:osp_precreate_thread()) cfs_fail_timeout id 2107 awake [ 3819.620513] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3819.791568] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3821.765614] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing set_default_debug -1 all [ 3823.662988] Lustre: DEBUG MARKER: oleg208-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3836.195875] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3836.200306] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 192.168.202.108@tcp (at 0@lo) [ 3836.200376] LustreError: lustre-OST0000-osc-MDT0001: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3844.905806] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid in FULL state after 20 sec [ 3846.171796] Lustre: DEBUG MARKER: oleg208-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff9858c4345800.ost_server_uuid 50 [ 3846.680037] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff9858c4345800.ost_server_uuid in FULL state after 0 sec [ 3851.747181] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3851.748257] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3851.751831] Lustre: Skipped 1 previous similar message [ 3851.756934] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3851.759846] Lustre: Skipped 2 previous similar messages [ 3853.738698] LustreError: 221534:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 3853.742693] LustreError: 221534:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 3853.809183] Lustre: server umount lustre-OST0000 complete [ 3856.865976] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3856.870379] Lustre: Skipped 2 previous similar messages [ 3861.306525] LustreError: 221737:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup [ 3861.309177] LustreError: 221737:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 3861.337605] Lustre: server umount lustre-MDT0000 complete [ 3861.984134] LustreError: 217796:0:(ldlm_lib.c:1113:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3861.988020] LustreError: 217796:0:(ldlm_lib.c:1113:target_handle_connect()) Skipped 3 previous similar messages [ 3862.671146] LustreError: 217775:0:(ldlm_lockd.c:2550:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1755043227 with bad export cookie 12605988349152340878 [ 3862.674254] LustreError: MGC192.168.202.108@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3862.676434] LustreError: 217775:0:(ldlm_lockd.c:2550:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 3862.827639] Lustre: server umount lustre-MDT0001 complete [ 3867.638854] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing unload_modules_local [ 3868.831711] Key type lgssc unregistered [ 3868.970564] LNet: 222463:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3869.989326] LNet: Removed LNI 192.168.202.108@tcp [ 3870.313772] Key type .llcrypt unregistered [ 3870.315233] Key type ._llcrypt unregistered [ 3882.743642] Key type ._llcrypt registered [ 3882.744469] Key type .llcrypt registered [ 3882.784639] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing load_modules_local [ 3883.172628] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3883.217829] alg: No test for adler32 (adler32-zlib) [ 3884.073763] Lustre: Lustre: Build Version: 2.16.56_30_ga643f08 [ 3884.171090] LNet: Added LNI 192.168.202.108@tcp [8/256/0/180] [ 3884.173208] LNet: Accept secure, port 988 [ 3885.759147] Key type lgssc registered [ 3886.123898] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3889.077891] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3889.086995] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3890.236905] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3891.499106] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing set_default_debug -1 all [ 3894.136843] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3894.247276] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3895.488575] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing set_default_debug -1 all [ 3896.400101] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 3898.518829] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3898.635227] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3900.357576] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing set_default_debug -1 all [ 3900.711818] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49699 to 0x280000401:49729) [ 3903.188383] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3903.214455] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3903.277396] Lustre: lustre-OST0001: new disk, initializing [ 3903.279304] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 3903.313114] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 3905.030607] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing set_default_debug -1 all [ 3910.425620] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 3910.659571] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 3910.666207] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 3910.694663] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 3915.154411] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 3930.081727] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3930.084040] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3930.087617] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3931.106093] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3931.106227] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3931.110661] Lustre: Skipped 2 previous similar messages [ 3931.113703] Lustre: Skipped 2 previous similar messages [ 3933.634322] LustreError: 227485:0:(obd_class.h:479:obd_check_dev()) Device 28 not setup [ 3933.676069] Lustre: server umount lustre-MDT0000 complete [ 3934.954209] LustreError: 225240:0:(ldlm_lockd.c:2550:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1755043300 with bad export cookie 4468770720425830996 [ 3934.957992] LustreError: MGC192.168.202.108@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3934.958710] LustreError: 225240:0:(ldlm_lockd.c:2550:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 3935.022245] LustreError: 227686:0:(obd_class.h:479:obd_check_dev()) Device 27 not setup [ 3935.023963] LustreError: 227686:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 3935.108981] Lustre: server umount lustre-MDT0001 complete [ 3946.610706] LustreError: 227886:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 3946.612443] LustreError: 227886:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 3946.633405] Lustre: server umount lustre-OST0000 complete [ 3958.128897] LustreError: 228088:0:(obd_class.h:479:obd_check_dev()) Device 2 not setup [ 3958.130730] LustreError: 228088:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 3958.203035] Lustre: server umount lustre-OST0001 complete [ 3959.271211] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro [ 3959.314915] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3959.691466] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro [ 3959.727639] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3960.118665] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: ,errors=remount-ro [ 3960.147800] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3960.541110] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: ,errors=remount-ro [ 3960.574943] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3965.388693] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing load_modules_local [ 3968.741388] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3968.894271] LustreError: 229485:0:(ldlm_lib.c:1113:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3968.900429] LustreError: 229485:0:(ldlm_lib.c:1113:target_handle_connect()) Skipped 2 previous similar messages [ 3968.917836] Lustre: mylustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3970.251272] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing set_default_debug -1 all [ 3973.232817] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3974.778504] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing set_default_debug -1 all [ 3975.815150] Lustre: 230584:0:(mgs_llog.c:1345:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 3978.178439] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3978.313685] Lustre: mylustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3978.317588] Lustre: Skipped 1 previous similar message [ 3979.362681] LustreError: 230939:0:(ldlm_lib.c:1113:target_handle_connect()) mylustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3979.366577] Lustre: mylustre-OST0000: new connection from mylustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49732 to 0x280000401:49761) [ 3980.257240] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing set_default_debug -1 all [ 3983.256333] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3985.405421] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing set_default_debug -1 all [ 3988.670944] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 3990.016608] Lustre: 232414:0:(mgs_llog.c:1345:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 3999.716703] Lustre: mylustre-MDT0000-lwp-OST0001: Connection to mylustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3999.718499] Lustre: mylustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3999.720511] Lustre: Skipped 2 previous similar messages [ 3999.722432] Lustre: Skipped 2 previous similar messages [ 4004.319965] LustreError: mylustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4004.322183] Lustre: mylustre-MDT0000-osp-MDT0001: Connection to mylustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4004.325875] Lustre: mylustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4004.327463] Lustre: Skipped 1 previous similar message [ 4005.504840] LustreError: 232719:0:(obd_class.h:479:obd_check_dev()) Device 10 not setup [ 4005.506606] LustreError: 232719:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 4005.539458] Lustre: server umount mylustre-MDT0000 complete [ 4006.773221] LustreError: 231745:0:(ldlm_lockd.c:2550:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1755043371 with bad export cookie 4468770720425833789 [ 4006.776775] LustreError: 231745:0:(ldlm_lockd.c:2550:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 4006.777363] LustreError: MGC192.168.202.108@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4006.907558] Lustre: server umount mylustre-MDT0001 complete [ 4018.163052] LustreError: 233121:0:(obd_class.h:479:obd_check_dev()) Device 23 not setup [ 4018.165390] LustreError: 233121:0:(obd_class.h:479:obd_check_dev()) Skipped 15 previous similar messages [ 4018.182346] Lustre: server umount mylustre-OST0000 complete [ 4029.767118] Lustre: server umount mylustre-OST0001 complete [ 4030.851481] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro [ 4030.892037] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4031.256305] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro [ 4031.284592] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4031.627346] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: ,errors=remount-ro [ 4031.649585] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4031.988044] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: ,errors=remount-ro [ 4032.018146] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4036.841529] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing load_modules_local [ 4040.391443] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4040.566091] LustreError: 234721:0:(ldlm_lib.c:1113:target_handle_connect()) tfs-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4040.592812] Lustre: tfs-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4040.594666] Lustre: Skipped 1 previous similar message [ 4041.978281] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing set_default_debug -1 all [ 4044.912256] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4046.412611] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing set_default_debug -1 all [ 4047.445797] Lustre: 235818:0:(mgs_llog.c:1345:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 4049.775190] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4050.913931] LustreError: 236357:0:(ldlm_lib.c:1113:target_handle_connect()) tfs-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4050.920077] Lustre: tfs-OST0000: new connection from tfs-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49763 to 0x280000401:49793) [ 4051.882248] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing set_default_debug -1 all [ 4054.841174] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4056.827475] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing set_default_debug -1 all [ 4059.963933] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 4061.194156] Lustre: 237653:0:(mgs_llog.c:1345:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 4070.879978] LustreError: tfs-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4070.882419] Lustre: tfs-MDT0000-osp-MDT0001: Connection to tfs-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4070.885859] Lustre: tfs-MDT0000: Not available for connect from 0@lo (stopping) [ 4070.887621] Lustre: Skipped 3 previous similar messages [ 4074.238995] LustreError: 237903:0:(obd_class.h:479:obd_check_dev()) Device 10 not setup [ 4074.240848] LustreError: 237903:0:(obd_class.h:479:obd_check_dev()) Skipped 9 previous similar messages [ 4074.270500] Lustre: server umount tfs-MDT0000 complete [ 4075.528700] LustreError: 235442:0:(ldlm_lockd.c:2550:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1755043440 with bad export cookie 4468770720425835455 [ 4075.534572] LustreError: MGC192.168.202.108@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4075.534781] LustreError: 235442:0:(ldlm_lockd.c:2550:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 4075.688489] Lustre: server umount tfs-MDT0001 complete [ 4087.229207] Lustre: server umount tfs-OST0000 complete [ 4098.816337] Lustre: server umount tfs-OST0001 complete [ 4099.919490] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro [ 4099.957472] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4100.340469] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro [ 4100.373064] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4100.750868] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: ,errors=remount-ro [ 4100.777372] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4101.167752] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: ,errors=remount-ro [ 4101.217788] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4106.165239] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing load_modules_local [ 4110.049582] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4110.223411] LustreError: 239905:0:(ldlm_lib.c:1113:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4110.248537] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4110.252025] Lustre: Skipped 3 previous similar messages [ 4111.710938] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing set_default_debug -1 all [ 4114.873713] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4116.365732] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing set_default_debug -1 all [ 4117.490127] Lustre: 241003:0:(mgs_llog.c:1345:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 4120.325114] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4122.722753] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing set_default_debug -1 all [ 4125.607199] LustreError: 242072:0:(ldlm_lib.c:1113:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4126.176170] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4128.277188] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing set_default_debug -1 all [ 4135.657720] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49763 to 0x280000401:49825) [ 4142.313778] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 4143.553764] Lustre: 242859:0:(mgs_llog.c:1345:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 4151.142895] Lustre: DEBUG MARKER: == conf-sanity test 104a: Make sure user defined options are reflected in mount ========================================================== 20:05:16 (1755043516) [ 4166.112111] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4166.114625] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4166.118506] Lustre: Skipped 3 previous similar messages [ 4166.120408] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4166.122383] Lustre: Skipped 4 previous similar messages [ 4177.375092] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 4177.417101] LustreError: 243228:0:(obd_class.h:479:obd_check_dev()) Device 10 not setup [ 4177.420902] LustreError: 243228:0:(obd_class.h:479:obd_check_dev()) Skipped 25 previous similar messages [ 4177.458513] Lustre: server umount lustre-MDT0000 complete [ 4178.683086] LustreError: 239879:0:(ldlm_lockd.c:2550:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1755043543 with bad export cookie 4468770720425836897 [ 4178.686453] LustreError: MGC192.168.202.108@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4178.686670] LustreError: 239879:0:(ldlm_lockd.c:2550:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 4181.985229] LustreError: 242600:0:(ldlm_lib.c:1113:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4181.996037] LustreError: 242600:0:(ldlm_lib.c:1113:target_handle_connect()) Skipped 1 previous similar message [ 4184.033850] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 4184.036025] Lustre: Skipped 14 previous similar messages [ 4184.910341] Lustre: server umount lustre-MDT0001 complete [ 4192.312255] Lustre: server umount lustre-OST0000 complete [ 4199.728885] Lustre: server umount lustre-OST0001 complete [ 4201.815446] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing set_hostid [ 4205.314798] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing load_modules_local [ 4209.268124] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4211.718615] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4213.777328] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4215.986477] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4218.542041] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4218.565474] LDISKFS-fs (dm-0): Mount option "noacl" will be removed by 3.5 [ 4218.565474] Contact linux-ldiskfs@vger.kernel.org if you think we should keep it. [ 4218.565474] [ 4218.571314] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: acl,user_xattr,errors=remount-ro,noacl,no_mbcache,nodelalloc [ 4218.654740] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 4218.664927] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 4218.700753] Lustre: lustre-MDT0000: new disk, initializing [ 4218.726488] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4218.731448] Lustre: Skipped 3 previous similar messages [ 4218.739284] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 4219.988498] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing set_default_debug -1 all [ 4224.510993] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4224.540280] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4224.578717] Lustre: 245997:0:(mgs_llog.c:1345:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 4224.606533] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 4224.609791] Lustre: Skipped 1 previous similar message [ 4224.647920] Lustre: lustre-MDT0001: new disk, initializing [ 4224.690521] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 4224.694342] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 4226.039583] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing set_default_debug -1 all [ 4230.341117] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4230.364365] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4230.461506] Lustre: lustre-OST0000: new disk, initializing [ 4230.463558] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 4231.540822] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 4231.544678] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 4231.573842] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 4232.300493] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing set_default_debug -1 all [ 4236.499572] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4236.525215] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4236.579297] Lustre: lustre-OST0001: new disk, initializing [ 4236.581101] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 4237.770842] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 4237.776248] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 4237.815409] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 4238.477522] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing set_default_debug -1 all [ 4243.057634] Lustre: DEBUG MARKER: == conf-sanity test 104b: Mount uses last flock argument ========================================================== 20:06:48 (1755043608) [ 4246.927939] Lustre: DEBUG MARKER: == conf-sanity test 105: check file creation for ro and rw bind mnt pt ========================================================== 20:06:52 (1755043612) [ 4262.368369] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4262.368803] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4262.370685] LustreError: Skipped 1 previous similar message [ 4262.373946] Lustre: Skipped 5 previous similar messages [ 4262.374607] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4262.378586] Lustre: Skipped 1 previous similar message [ 4264.554130] LustreError: 249390:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 4264.555926] LustreError: 249390:0:(obd_class.h:479:obd_check_dev()) Skipped 25 previous similar messages [ 4264.571547] Lustre: server umount lustre-OST0000 complete [ 4268.514698] LustreError: 247724:0:(ldlm_lib.c:1113:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4268.525089] LustreError: 247724:0:(ldlm_lib.c:1113:target_handle_connect()) Skipped 2 previous similar messages [ 4280.287102] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 4280.350893] Lustre: server umount lustre-MDT0000 complete [ 4281.615294] LustreError: 245989:0:(ldlm_lockd.c:2550:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1755043646 with bad export cookie 4468770720425838227 [ 4281.618114] LustreError: 245989:0:(ldlm_lockd.c:2550:ldlm_cancel_handler()) Skipped 1 previous similar message [ 4281.619763] LustreError: MGC192.168.202.108@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4287.829006] Lustre: server umount lustre-MDT0001 complete [ 4292.759535] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing unload_modules_local [ 4302.443577] Lustre: server umount lustre-OST0001 complete [ 4304.453308] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing set_hostid [ 4310.944127] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing load_modules_local [ 4314.344507] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4316.573480] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4318.544434] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4320.604173] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4322.748207] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4322.771298] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4322.844700] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 4322.856093] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 4322.890538] Lustre: lustre-MDT0000: new disk, initializing [ 4322.917254] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 4324.160272] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing set_default_debug -1 all [ 4328.129318] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4328.154309] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4328.184383] Lustre: 252840:0:(mgs_llog.c:1345:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 4328.284766] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 4329.520742] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing set_default_debug -1 all [ 4332.620300] Lustre: DEBUG MARKER: oleg208-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 4333.742558] Lustre: DEBUG MARKER: oleg208-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 4336.039462] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4336.069591] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4338.015085] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing set_default_debug -1 all [ 4338.040058] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 4338.080293] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 4341.077655] Lustre: DEBUG MARKER: oleg208-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4348.385710] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4348.386287] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4348.387733] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4348.387739] Lustre: Skipped 14 previous similar messages [ 4348.389167] Lustre: Skipped 6 previous similar messages [ 4348.391403] LustreError: Skipped 2 previous similar messages [ 4349.543213] Lustre: server umount lustre-OST0000 complete [ 4365.279077] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 4365.335559] Lustre: server umount lustre-MDT0000 complete [ 4366.544799] LustreError: 252831:0:(ldlm_lockd.c:2550:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1755043731 with bad export cookie 4468770720425839697 [ 4366.547631] LustreError: 252831:0:(ldlm_lockd.c:2550:ldlm_cancel_handler()) Skipped 1 previous similar message [ 4366.548460] LustreError: MGC192.168.202.108@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4366.685847] Lustre: server umount lustre-MDT0001 complete [ 4371.445195] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing unload_modules_local [ 4372.462950] Key type lgssc unregistered [ 4372.608321] LNet: 256192:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4373.669355] LNet: Removed LNI 192.168.202.108@tcp [ 4373.977205] Key type .llcrypt unregistered [ 4373.978996] Key type ._llcrypt unregistered [ 4384.223399] Key type ._llcrypt registered [ 4384.224482] Key type .llcrypt registered [ 4384.257305] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing load_modules_local [ 4384.572443] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4384.615239] alg: No test for adler32 (adler32-zlib) [ 4385.472093] Lustre: Lustre: Build Version: 2.16.56_30_ga643f08 [ 4385.554027] LNet: Added LNI 192.168.202.108@tcp [8/256/0/180] [ 4385.555768] LNet: Accept secure, port 988 [ 4387.135106] Key type lgssc registered [ 4387.505477] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4390.057805] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4390.065812] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4391.252191] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4392.621653] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing set_default_debug -1 all [ 4395.583846] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4395.719803] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4397.102547] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing set_default_debug -1 all [ 4398.923693] Lustre: DEBUG MARKER: oleg208-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 4400.074936] Lustre: DEBUG MARKER: oleg208-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 4402.605140] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4402.762431] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4404.804652] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing set_default_debug -1 all [ 4406.688345] Lustre: DEBUG MARKER: oleg208-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4407.156571] Lustre: Setting parameter lustre-OST0000.ost.unknown_param=50 in log lustre-OST0000 [ 4411.367709] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4411.377881] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4411.378439] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4414.124196] LustreError: 259879:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 4414.193377] Lustre: server umount lustre-OST0000 complete [ 4416.482682] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4416.483205] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4416.484336] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4416.502275] Lustre: Skipped 1 previous similar message [ 4421.601216] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4421.604670] Lustre: Skipped 2 previous similar messages [ 4421.709876] LustreError: 260084:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup [ 4421.716290] LustreError: 260084:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 4421.745946] Lustre: server umount lustre-MDT0000 complete [ 4423.374084] LustreError: 258312:0:(ldlm_lockd.c:2550:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1755043788 with bad export cookie 443345915620580772 [ 4423.379658] LustreError: 258312:0:(ldlm_lockd.c:2550:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 4423.380841] LustreError: MGC192.168.202.108@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4423.460369] LustreError: 260285:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [ 4423.462031] LustreError: 260285:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 4423.546698] Lustre: server umount lustre-MDT0001 complete [ 4427.675791] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing unload_modules_local [ 4429.000279] Key type lgssc unregistered [ 4429.176610] LNet: 260811:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4430.245556] LNet: Removed LNI 192.168.202.108@tcp [ 4430.723885] Key type .llcrypt unregistered [ 4430.726113] Key type ._llcrypt unregistered [ 4437.708294] Key type ._llcrypt registered [ 4437.709402] Key type .llcrypt registered [ 4437.752515] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing load_modules_local [ 4438.207068] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4438.275069] alg: No test for adler32 (adler32-zlib) [ 4439.140180] Lustre: Lustre: Build Version: 2.16.56_30_ga643f08 [ 4439.229867] LNet: Added LNI 192.168.202.108@tcp [8/256/0/180] [ 4439.231200] LNet: Accept secure, port 988 [ 4440.815204] Key type lgssc registered [ 4441.200542] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4444.431045] Lustre: lustre-OST0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4444.437610] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4461.919298] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 4467.039347] LustreError: 261953:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.202.108@tcp: failed processing log, type 4: rc = -110 [ 4497.759265] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4499.816564] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing set_default_debug -1 all [ 4501.920411] Lustre: DEBUG MARKER: oleg208-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4502.631308] LustreError: 262612:0:(obd_class.h:479:obd_check_dev()) Device 1 not setup [ 4502.713634] Lustre: server umount lustre-OST0000 complete [ 4508.819475] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing unload_modules_local [ 4509.941445] Key type lgssc unregistered [ 4510.082439] LNet: 263443:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4511.141456] LNet: Removed LNI 192.168.202.108@tcp [ 4511.485469] Key type .llcrypt unregistered [ 4511.486423] Key type ._llcrypt unregistered [ 4524.884069] Key type ._llcrypt registered [ 4524.885176] Key type .llcrypt registered [ 4524.917874] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing load_modules_local [ 4525.214438] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4525.237021] alg: No test for adler32 (adler32-zlib) [ 4526.096597] Lustre: Lustre: Build Version: 2.16.56_30_ga643f08 [ 4526.173306] LNet: Added LNI 192.168.202.108@tcp [8/256/0/180] [ 4526.174815] LNet: Accept secure, port 988 [ 4527.759178] Key type lgssc registered [ 4528.092715] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4532.423038] loop0: detected capacity change from 0 to 204800000 [ 4532.485962] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 4532.489662] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 4532.613549] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 4532.618408] blk_update_request: operation not supported error, dev loop0, sector 160240 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 4532.641720] blk_update_request: operation not supported error, dev loop0, sector 160344 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 4532.661030] blk_update_request: operation not supported error, dev loop0, sector 168552 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 4532.675072] blk_update_request: operation not supported error, dev loop0, sector 173280 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 4532.697489] blk_update_request: operation not supported error, dev loop0, sector 176912 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 4532.726704] blk_update_request: operation not supported error, dev loop0, sector 185112 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 4532.750856] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4533.074073] loop0: detected capacity change from 0 to 204800000 [ 4533.128030] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 4533.344617] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4533.711159] loop0: detected capacity change from 0 to 204800000 [ 4533.869595] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4534.284161] loop0: detected capacity change from 0 to 204800000 [ 4534.538376] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4534.926846] loop0: detected capacity change from 0 to 204800000 [ 4534.938066] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 4536.543045] loop0: detected capacity change from 0 to 204800000 [ 4536.556417] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 4537.976209] loop0: detected capacity change from 0 to 204800000 [ 4537.989445] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 4539.330050] loop0: detected capacity change from 0 to 204800000 [ 4539.337049] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 4540.730476] loop0: detected capacity change from 0 to 204800000 [ 4540.785344] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4540.812497] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4540.819804] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4540.844980] Lustre: 266808:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 4540.848183] Lustre: 266808:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index 'nodemap' with [0x200000003:0x3:0x0]: rc = 0 [ 4540.851647] Lustre: 266808:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x2010000-MDT0000' with [0x200000005:0x3:0x0]: rc = 0 [ 4540.856680] Lustre: 266808:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x1010000-MDT0000' with [0x200000005:0x2:0x0]: rc = 0 [ 4540.860138] Lustre: 266808:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000003:0xb:0x0]: rc = 0 [ 4540.863066] Lustre: 266808:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000003:0xc:0x0]: rc = 0 [ 4540.865991] Lustre: 266808:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x10000-MDT0000' with [0x200000005:0x1:0x0]: rc = 0 [ 4540.869052] Lustre: 266808:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000003:0xa:0x0]: rc = 0 [ 4540.871896] Lustre: 266808:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x20000' with [0x200000006:0x20000:0x0]: rc = 0 [ 4540.874837] Lustre: 266808:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x1020000' with [0x200000006:0x1020000:0x0]: rc = 0 [ 4540.877957] Lustre: 266808:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x2020000' with [0x200000006:0x2020000:0x0]: rc = 0 [ 4540.881204] Lustre: 266808:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000006:0x1010000:0x0]: rc = 0 [ 4540.884362] Lustre: 266808:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000006:0x2010000:0x0]: rc = 0 [ 4540.887335] Lustre: 266808:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000006:0x10000:0x0]: rc = 0 [ 4543.402028] LustreError: 267073:0:(obd_class.h:479:obd_check_dev()) Device 2 not setup [ 4543.444755] Lustre: server umount MGS complete [ 4543.719749] loop0: detected capacity change from 0 to 204800000 [ 4543.753220] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4543.901036] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4543.916177] Lustre: lustre-MDT0000: Aborting client recovery [ 4544.211122] loop1: detected capacity change from 0 to 204800000 [ 4544.245917] LDISKFS-fs (loop1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4544.278620] LDISKFS-fs (loop1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4544.306525] Lustre: 267241:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 4544.309397] Lustre: 267241:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0001: restore index '0x2010000-MDT0001' with [0x200000005:0x3:0x0]: rc = 0 [ 4544.312422] Lustre: 267241:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0001: restore index '0x10000-MDT0001' with [0x200000005:0x1:0x0]: rc = 0 [ 4544.315453] Lustre: 267241:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0001: restore index '0x1010000-MDT0001' with [0x200000005:0x2:0x0]: rc = 0 [ 4544.318696] Lustre: 267241:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0001: restore index '0x10000' with [0x200000003:0x4:0x0]: rc = 0 [ 4544.321429] Lustre: 267241:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0001: restore index '0x2010000' with [0x200000003:0x6:0x0]: rc = 0 [ 4544.324413] Lustre: 267241:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0001: restore index '0x1010000' with [0x200000003:0x5:0x0]: rc = 0 [ 4544.330390] Lustre: Found index 1 for lustre-MDT0001, updating log [ 4544.422407] Lustre: lustre-MDT0001: Aborting client recovery [ 4544.724382] loop2: detected capacity change from 0 to 204800000 [ 4544.757804] LDISKFS-fs (loop2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4544.787455] LDISKFS-fs (loop2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4544.809252] Lustre: 267326:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 4544.812230] Lustre: 267326:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0000: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 4544.815889] Lustre: 267326:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0000: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 4544.818730] Lustre: 267326:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0000: restore index '0x2020000-OST0000' with [0x200000005:0x3:0x0]: rc = 0 [ 4544.821703] Lustre: 267326:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0000: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 4544.824754] Lustre: 267326:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0000: restore index '0x1020000-OST0000' with [0x200000005:0x2:0x0]: rc = 0 [ 4544.830061] Lustre: 267326:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0000: restore index '0x20000-OST0000' with [0x200000005:0x1:0x0]: rc = 0 [ 4544.879771] Lustre: Found index 0 for lustre-OST0000, updating log [ 4544.908801] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4544.910535] Lustre: Skipped 1 previous similar message [ 4544.915601] Lustre: lustre-OST0000: Aborting recovery [ 4545.214909] loop3: detected capacity change from 0 to 204800000 [ 4545.255986] LDISKFS-fs (loop3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4545.286576] LDISKFS-fs (loop3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4545.307254] Lustre: 267412:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 4545.310043] Lustre: 267412:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0001: restore index '0x1020000-OST0001' with [0x200000005:0x2:0x0]: rc = 0 [ 4545.312814] Lustre: 267412:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0001: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 4545.315624] Lustre: 267412:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0001: restore index '0x2020000-OST0001' with [0x200000005:0x3:0x0]: rc = 0 [ 4545.318699] Lustre: 267412:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0001: restore index '0x20000-OST0001' with [0x200000005:0x1:0x0]: rc = 0 [ 4545.321684] Lustre: 267412:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0001: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 4545.324362] Lustre: 267412:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0001: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 4550.631736] LustreError: 267184:0:(lod_dev.c:510:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 6, retries 0, failed: rc = -108 [ 4554.723616] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 4554.724292] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 4554.724673] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:5 to 0x280000400:33) [ 4554.729758] Lustre: lustre-OST0001: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:5 to 0x2c0000400:33) [ 4554.739226] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000bd0 [ 4554.740208] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000bd0 [ 4555.817792] LustreError: 267152:0:(mdt_identity.c:95:mdt_identity_do_upcall()) lustre-MDT0000: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0000 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0000/identity_upcall, time 237us: rc = -2 [ 4570.080769] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4570.080948] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4570.080948] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4570.081244] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4570.090097] Lustre: Skipped 2 previous similar messages [ 4573.306863] LustreError: 267768:0:(obd_class.h:479:obd_check_dev()) Device 10 not setup [ 4573.308411] LustreError: 267768:0:(obd_class.h:479:obd_check_dev()) Skipped 1 previous similar message [ 4573.343626] Lustre: server umount lustre-MDT0000 complete [ 4573.634664] LustreError: 267250:0:(ldlm_lockd.c:2550:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1755043938 with bad export cookie 13845579514059946889 [ 4573.636044] LustreError: MGC192.168.202.108@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4573.637344] LustreError: 267250:0:(ldlm_lockd.c:2550:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 4575.199586] LustreError: 267149:0:(ldlm_lib.c:1113:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4575.199926] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4575.200178] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 4575.200185] Lustre: Skipped 3 previous similar messages [ 4575.203232] LustreError: 267149:0:(ldlm_lib.c:1113:target_handle_connect()) Skipped 3 previous similar messages [ 4575.205679] Lustre: Skipped 1 previous similar message [ 4578.207463] LustreError: 267149:0:(ldlm_lib.c:1113:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4578.207600] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 4578.211333] LustreError: 267149:0:(ldlm_lib.c:1113:target_handle_connect()) Skipped 1 previous similar message [ 4578.216675] Lustre: Skipped 2 previous similar messages [ 4579.843444] LustreError: 267820:0:(obd_class.h:479:obd_check_dev()) Device 18 not setup [ 4579.846569] LustreError: 267820:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 4579.932755] Lustre: server umount lustre-MDT0001 complete [ 4586.346482] LustreError: 267871:0:(obd_class.h:479:obd_check_dev()) Device 23 not setup [ 4586.348352] LustreError: 267871:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 4586.370760] Lustre: server umount lustre-OST0000 complete [ 4592.745772] LustreError: 267922:0:(obd_class.h:479:obd_check_dev()) Device 2 not setup [ 4592.748823] LustreError: 267922:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 4592.853544] Lustre: server umount lustre-OST0001 complete [ 4595.501972] Lustre: DEBUG MARKER: == conf-sanity test 109a: test lctl clear_conf fsname ==== 20:12:41 (1755043961) [ 4603.734526] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing set_hostid [ 4606.394924] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing load_modules_local [ 4609.517672] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4611.633106] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4613.778830] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4615.867625] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4618.380450] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4618.410554] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4618.506042] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 4618.516581] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 4618.555484] Lustre: lustre-MDT0000: new disk, initializing [ 4618.580484] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4618.583024] Lustre: Skipped 1 previous similar message [ 4618.589087] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 4619.998771] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing set_default_debug -1 all [ 4624.434607] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4624.461600] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4624.490326] Lustre: 271449:0:(mgs_llog.c:1345:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 4624.514532] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 4624.516269] Lustre: Skipped 1 previous similar message [ 4624.555855] Lustre: lustre-MDT0001: new disk, initializing [ 4624.592553] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4624.605688] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 4624.608871] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 4625.998427] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing set_default_debug -1 all [ 4629.360840] Lustre: DEBUG MARKER: oleg208-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 4630.688682] Lustre: DEBUG MARKER: oleg208-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 4633.628865] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4633.656156] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4633.785780] Lustre: lustre-OST0000: new disk, initializing [ 4633.787775] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 4633.817803] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4635.722523] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing set_default_debug -1 all [ 4636.937490] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 4636.942983] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 4636.997771] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 4639.277881] Lustre: DEBUG MARKER: oleg208-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4641.927183] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [ 4647.625183] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [ 4657.428701] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [ 4661.986866] Lustre: Modifying parameter lustre-client.llite.max_read_ahead_mb=64 in log lustre-client [ 4698.596954] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4698.597720] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4698.601267] LustreError: Skipped 1 previous similar message [ 4698.609865] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4698.616978] Lustre: Skipped 1 previous similar message [ 4703.716364] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4703.720488] Lustre: Skipped 1 previous similar message [ 4704.493518] LustreError: 275136:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 4704.497494] LustreError: 275136:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 4704.598526] Lustre: server umount lustre-OST0000 complete [ 4706.784418] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4706.790416] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4706.795524] Lustre: Skipped 1 previous similar message [ 4712.491350] Lustre: server umount lustre-MDT0000 complete [ 4713.952665] LustreError: 272230:0:(ldlm_lib.c:1113:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4713.962649] LustreError: 272230:0:(ldlm_lib.c:1113:target_handle_connect()) Skipped 1 previous similar message [ 4714.277052] LustreError: 271441:0:(ldlm_lockd.c:2550:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1755044079 with bad export cookie 13845579514059949598 [ 4714.281929] LustreError: MGC192.168.202.108@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4714.283494] LustreError: 271441:0:(ldlm_lockd.c:2550:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 4714.467084] Lustre: server umount lustre-MDT0001 complete [ 4718.543900] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4720.243691] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing set_default_debug -1 all [ 4722.597198] LustreError: 276472:0:(obd_class.h:479:obd_check_dev()) Device 2 not setup [ 4722.600735] LustreError: 276472:0:(obd_class.h:479:obd_check_dev()) Skipped 17 previous similar messages [ 4722.635377] Lustre: server umount MGS complete [ 4728.003155] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4728.207899] LustreError: 277096:0:(ldlm_lib.c:1113:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4728.216510] LustreError: 277108:0:(qmt_pool.c:1513:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [ 4728.249339] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4729.835811] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing set_default_debug -1 all [ 4733.097796] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4734.936335] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing set_default_debug -1 all [ 4737.213524] Lustre: DEBUG MARKER: oleg208-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 4738.430623] Lustre: DEBUG MARKER: oleg208-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 4741.391263] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4743.716265] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing set_default_debug -1 all [ 4746.077476] Lustre: DEBUG MARKER: oleg208-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4779.489970] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4779.490218] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4779.493862] LustreError: Skipped 1 previous similar message [ 4779.497748] Lustre: Skipped 1 previous similar message [ 4779.498416] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4779.501278] Lustre: Skipped 4 previous similar messages [ 4781.163097] LustreError: 279754:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 4781.164956] LustreError: 279754:0:(obd_class.h:479:obd_check_dev()) Skipped 1 previous similar message [ 4781.222693] Lustre: server umount lustre-OST0000 complete [ 4784.609145] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4788.500615] Lustre: server umount lustre-MDT0000 complete [ 4789.729268] LustreError: 277092:0:(ldlm_lib.c:1113:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4789.737027] LustreError: 277092:0:(ldlm_lib.c:1113:target_handle_connect()) Skipped 1 previous similar message [ 4789.756487] LustreError: 278702:0:(ldlm_lockd.c:2550:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1755044154 with bad export cookie 13845579514059951411 [ 4789.763046] LustreError: MGC192.168.202.108@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4789.765298] LustreError: 278702:0:(ldlm_lockd.c:2550:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4789.929886] Lustre: server umount lustre-MDT0001 complete [ 4794.155720] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing unload_modules_local [ 4795.220869] Key type lgssc unregistered [ 4795.356378] LNet: 280684:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4796.389325] LNet: Removed LNI 192.168.202.108@tcp [ 4796.669235] Key type .llcrypt unregistered [ 4796.670272] Key type ._llcrypt unregistered [ 4809.339570] Key type ._llcrypt registered [ 4809.340828] Key type .llcrypt registered [ 4809.383250] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing set_hostid [ 4816.229793] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing load_modules_local [ 4816.571802] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4816.606211] alg: No test for adler32 (adler32-zlib) [ 4817.447646] Lustre: Lustre: Build Version: 2.16.56_30_ga643f08 [ 4817.528702] LNet: Added LNI 192.168.202.108@tcp [8/256/0/180] [ 4817.530347] LNet: Accept secure, port 988 [ 4819.111228] Key type lgssc registered [ 4819.555164] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4823.195945] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4825.399548] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4827.559392] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4829.736153] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4832.577877] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4832.601964] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4832.610284] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4833.696553] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 4833.706597] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 4833.741843] Lustre: lustre-MDT0000: new disk, initializing [ 4833.764104] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4833.769660] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 4835.003075] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing set_default_debug -1 all [ 4839.495036] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4839.517091] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4839.551992] Lustre: 284572:0:(mgs_llog.c:1345:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 4839.576443] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 4839.578850] Lustre: Skipped 1 previous similar message [ 4839.614083] Lustre: lustre-MDT0001: new disk, initializing [ 4839.648115] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4839.658884] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 4839.661668] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 4840.960284] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing set_default_debug -1 all [ 4844.622450] Lustre: DEBUG MARKER: oleg208-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 4846.172590] Lustre: DEBUG MARKER: oleg208-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 4848.504087] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4848.527697] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4848.614794] Lustre: lustre-OST0000: new disk, initializing [ 4848.616385] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 4848.640586] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4850.693064] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing set_default_debug -1 all [ 4853.762416] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 4853.766904] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 4853.816731] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 4854.325112] Lustre: DEBUG MARKER: oleg208-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4858.184060] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [ 4860.920153] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [ 4869.776787] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [ 4920.293296] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4920.293669] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4920.297061] LustreError: Skipped 1 previous similar message [ 4920.302418] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4920.304152] Lustre: Skipped 1 previous similar message [ 4921.326091] LustreError: 288111:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 4921.396763] Lustre: server umount lustre-OST0000 complete [ 4925.408920] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4925.409429] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4925.413268] Lustre: Skipped 2 previous similar messages [ 4925.416107] Lustre: Skipped 1 previous similar message [ 4928.763706] LustreError: 288313:0:(obd_class.h:479:obd_check_dev()) Device 22 not setup [ 4928.765655] LustreError: 288313:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 4928.794348] Lustre: server umount lustre-MDT0000 complete [ 4930.059054] LustreError: 287041:0:(ldlm_lockd.c:2550:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1755044295 with bad export cookie 9669291428627429457 [ 4930.062301] LustreError: MGC192.168.202.108@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4930.063395] LustreError: 287041:0:(ldlm_lockd.c:2550:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4930.126721] LustreError: 288515:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 4930.128878] LustreError: 288515:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 4930.213170] Lustre: server umount lustre-MDT0001 complete [ 4933.055679] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4934.321061] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing set_default_debug -1 all [ 4935.876585] LustreError: 289450:0:(obd_class.h:479:obd_check_dev()) Device 2 not setup [ 4935.878519] LustreError: 289450:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 4935.897267] Lustre: server umount MGS complete [ 4939.821476] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4939.980467] LustreError: 290084:0:(qmt_pool.c:1513:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [ 4940.000824] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4941.207584] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing set_default_debug -1 all [ 4944.288990] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4944.416720] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4945.855170] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing set_default_debug -1 all [ 4947.593222] Lustre: DEBUG MARKER: oleg208-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 4948.913551] Lustre: DEBUG MARKER: oleg208-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 4951.201550] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4953.377804] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing set_default_debug -1 all [ 4955.176733] Lustre: DEBUG MARKER: oleg208-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4990.433183] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4990.437997] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4990.445726] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4991.460596] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4993.965811] LustreError: 292732:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 4993.967876] LustreError: 292732:0:(obd_class.h:479:obd_check_dev()) Skipped 1 previous similar message [ 4994.031700] Lustre: server umount lustre-OST0000 complete [ 4995.552434] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4995.556586] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4995.563303] Lustre: Skipped 1 previous similar message [ 4995.567410] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4995.570873] Lustre: Skipped 2 previous similar messages [ 5001.574669] Lustre: server umount lustre-MDT0000 complete [ 5001.696337] LustreError: 290071:0:(ldlm_lib.c:1113:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5001.702665] LustreError: 290071:0:(ldlm_lib.c:1113:target_handle_connect()) Skipped 3 previous similar messages [ 5003.355424] LustreError: 292469:0:(ldlm_lockd.c:2550:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1755044368 with bad export cookie 9669291428627431284 [ 5003.359091] LustreError: 292469:0:(ldlm_lockd.c:2550:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 5003.359809] LustreError: MGC192.168.202.108@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5003.432874] LustreError: 293135:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [ 5003.436382] LustreError: 293135:0:(obd_class.h:479:obd_check_dev()) Skipped 9 previous similar messages [ 5003.521157] Lustre: server umount lustre-MDT0001 complete [ 5008.897757] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing unload_modules_local [ 5010.081330] Key type lgssc unregistered [ 5010.215325] LNet: 293661:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5011.237478] LNet: Removed LNI 192.168.202.108@tcp [ 5011.577971] Key type .llcrypt unregistered [ 5011.578922] Key type ._llcrypt unregistered [ 5019.723343] Key type ._llcrypt registered [ 5019.724454] Key type .llcrypt registered [ 5019.773276] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing set_hostid [ 5026.813726] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing load_modules_local [ 5027.152050] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5027.178092] alg: No test for adler32 (adler32-zlib) [ 5028.045062] Lustre: Lustre: Build Version: 2.16.56_30_ga643f08 [ 5028.135316] LNet: Added LNI 192.168.202.108@tcp [8/256/0/180] [ 5028.136771] LNet: Accept secure, port 988 [ 5029.719232] Key type lgssc registered [ 5030.240120] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5033.828876] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5036.725442] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5040.516398] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing load_modules_local [ 5043.610636] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5046.255214] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5046.278582] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5046.284732] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5047.365241] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 5047.376795] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 5047.412885] Lustre: lustre-MDT0000: new disk, initializing [ 5047.435421] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5047.440911] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 5048.758922] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing set_default_debug -1 all [ 5054.299796] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5054.336794] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5054.456463] Lustre: lustre-OST0000: new disk, initializing [ 5054.459917] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 5054.463290] Lustre: Skipped 1 previous similar message [ 5054.504304] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5054.652600] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 5054.655561] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 5054.667292] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 5056.542782] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing set_default_debug -1 all [ 5257.609757] LustreError: 298539:0:(obd_class.h:479:obd_check_dev()) Device 13 not setup [ 5257.748191] Lustre: server umount lustre-MDT0000 complete [ 5259.007622] LustreError: 298453:0:(ldlm_lockd.c:2550:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1755044624 with bad export cookie 17281385191713612106 [ 5259.012759] LustreError: MGC192.168.202.108@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5269.488381] LustreError: 298742:0:(obd_class.h:479:obd_check_dev()) Device 2 not setup [ 5269.490093] LustreError: 298742:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 5269.551949] Lustre: server umount lustre-OST0000 complete [ 5278.642129] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing set_hostid [ 5281.662240] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing load_modules_local [ 5284.966228] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5287.751703] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5290.161186] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5292.883684] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5295.444756] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5295.465086] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5295.541384] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 5295.550974] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 5295.584985] Lustre: lustre-MDT0000: new disk, initializing [ 5295.606867] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5295.612521] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 5296.742499] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing set_default_debug -1 all [ 5301.660134] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5301.703188] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5301.753727] Lustre: 301762:0:(mgs_llog.c:1345:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 5301.791707] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 5301.794101] Lustre: Skipped 1 previous similar message [ 5301.834572] Lustre: lustre-MDT0001: new disk, initializing [ 5301.887997] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5301.910182] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 5301.916143] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 5303.763039] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing set_default_debug -1 all [ 5307.021561] Lustre: DEBUG MARKER: oleg208-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 5308.352737] Lustre: DEBUG MARKER: oleg208-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 5310.822050] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5310.845230] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5310.963657] Lustre: lustre-OST0000: new disk, initializing [ 5310.965929] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 5311.000260] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5312.899632] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing set_default_debug -1 all [ 5316.564857] Lustre: DEBUG MARKER: oleg208-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 5318.234031] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 5318.645740] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 5318.648149] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 5318.678994] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 5319.379070] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 1 sec [ 5320.744393] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 5320.806842] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 5323.748077] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 5323.748808] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5323.754261] LustreError: Skipped 1 previous similar message [ 5323.762128] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5323.766791] Lustre: Skipped 1 previous similar message [ 5327.473986] LustreError: 304677:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 5327.477255] LustreError: 304677:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 5327.574904] Lustre: server umount lustre-OST0000 complete [ 5332.959718] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 5332.961513] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5332.964279] Lustre: Skipped 1 previous similar message [ 5332.965701] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5333.984155] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5333.984748] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5335.354791] LustreError: 304878:0:(obd_class.h:479:obd_check_dev()) Device 22 not setup [ 5335.356511] LustreError: 304878:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 5335.379076] Lustre: server umount lustre-MDT0000 complete [ 5336.514046] LustreError: 303496:0:(ldlm_lockd.c:2550:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1755044701 with bad export cookie 17281385191716552890 [ 5336.516662] LustreError: MGC192.168.202.108@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5336.516811] LustreError: 303496:0:(ldlm_lockd.c:2550:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 5336.653599] Lustre: server umount lustre-MDT0001 complete [ 5337.674652] Lustre: DEBUG MARKER: SKIP: conf-sanity test_111 skipping SLOW test 111 [ 5338.182254] Lustre: DEBUG MARKER: == conf-sanity test 112a: mount OST with no_create option ========================================================== 20:25:04 (1755044704) [ 5340.123706] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5340.290783] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5341.397560] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing set_default_debug -1 all [ 5343.961507] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5345.159356] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing set_default_debug -1 all [ 5346.713438] Lustre: DEBUG MARKER: oleg208-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 5347.735713] Lustre: DEBUG MARKER: oleg208-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 5349.685575] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5351.368715] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing set_default_debug -1 all [ 5353.089078] Lustre: DEBUG MARKER: oleg208-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 5355.609920] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5355.639916] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5355.701532] Lustre: lustre-OST0001: new disk, initializing [ 5355.703124] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 5357.498833] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing set_default_debug -1 all [ 5358.232097] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 5358.237402] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 5358.262848] LustreError: 308575:0:(osp_precreate.c:972:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [ 5359.333956] LustreError: 308579:0:(osp_precreate.c:972:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [ 5359.339917] LustreError: 308579:0:(osp_precreate.c:972:osp_precreate_cleanup_orphans()) Skipped 2 previous similar messages [ 5360.359388] LustreError: 308575:0:(osp_precreate.c:972:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [ 5360.365329] LustreError: 308575:0:(osp_precreate.c:972:osp_precreate_cleanup_orphans()) Skipped 1 previous similar message [ 5361.558261] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 5361.617551] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 5362.405784] LustreError: 308579:0:(osp_precreate.c:972:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [ 5362.413598] LustreError: 308579:0:(osp_precreate.c:972:osp_precreate_cleanup_orphans()) Skipped 3 previous similar messages [ 5362.912463] Lustre: DEBUG MARKER: oleg208-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff9858e0694000.ost_server_uuid 50 [ 5363.655636] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff9858e0694000.ost_server_uuid in FULL state after 0 sec [ 5365.317673] Lustre: DEBUG MARKER: oleg208-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff9858e0694000.ost_server_uuid 50 [ 5366.064866] Lustre: DEBUG MARKER: osc.lustre-OST0001-osc-ffff9858e0694000.ost_server_uuid in FULL state after 0 sec [ 5366.499995] LustreError: 308575:0:(osp_precreate.c:972:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [ 5366.504685] LustreError: 308575:0:(osp_precreate.c:972:osp_precreate_cleanup_orphans()) Skipped 7 previous similar messages [ 5367.527396] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 5378.016287] LustreError: lustre-OST0001-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 5378.016492] Lustre: lustre-OST0001-osc-MDT0000: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5378.018871] LustreError: Skipped 1 previous similar message [ 5378.019443] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 5378.023184] Lustre: Skipped 1 previous similar message [ 5378.028327] Lustre: Skipped 2 previous similar messages [ 5384.163379] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 5384.165124] Lustre: Skipped 4 previous similar messages [ 5392.351077] Lustre: lustre-OST0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 5392.366767] LustreError: 309422:0:(obd_class.h:479:obd_check_dev()) Device 25 not setup [ 5392.368355] LustreError: 309422:0:(obd_class.h:479:obd_check_dev()) Skipped 13 previous similar messages [ 5392.385764] Lustre: server umount lustre-OST0001 complete [ 5394.402154] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5394.402889] LustreError: 307233:0:(ldlm_lib.c:1113:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5394.403287] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5394.403296] Lustre: Skipped 4 previous similar messages [ 5394.405154] Lustre: Skipped 1 previous similar message [ 5394.408608] LustreError: 307233:0:(ldlm_lib.c:1113:target_handle_connect()) Skipped 3 previous similar messages [ 5399.523767] LustreError: 307234:0:(ldlm_lib.c:1113:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5399.532313] LustreError: 307234:0:(ldlm_lib.c:1113:target_handle_connect()) Skipped 1 previous similar message [ 5400.204923] Lustre: server umount lustre-OST0000 complete [ 5404.640858] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5408.074372] LustreError: 309828:0:(obd_class.h:479:obd_check_dev()) Device 28 not setup [ 5408.078993] LustreError: 309828:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 5408.114651] Lustre: server umount lustre-MDT0000 complete [ 5409.660749] LustreError: 305677:0:(ldlm_lockd.c:2550:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1755044774 with bad export cookie 17281385191716553758 [ 5409.666368] LustreError: 305677:0:(ldlm_lockd.c:2550:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 5409.666479] LustreError: MGC192.168.202.108@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5409.862244] Lustre: server umount lustre-MDT0001 complete [ 5414.979149] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing unload_modules_local [ 5415.983613] Key type lgssc unregistered [ 5416.102333] LNet: 310555:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5417.125545] LNet: Removed LNI 192.168.202.108@tcp [ 5417.504978] Key type .llcrypt unregistered [ 5417.506271] Key type ._llcrypt unregistered [ 5428.845206] Key type ._llcrypt registered [ 5428.847971] Key type .llcrypt registered [ 5428.914899] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing load_modules_local [ 5429.331381] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5429.378142] alg: No test for adler32 (adler32-zlib) [ 5430.231361] Lustre: Lustre: Build Version: 2.16.56_30_ga643f08 [ 5430.316462] LNet: Added LNI 192.168.202.108@tcp [8/256/0/180] [ 5430.317824] LNet: Accept secure, port 988 [ 5431.895207] Key type lgssc registered [ 5432.415153] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5435.597418] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5435.606958] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5436.861491] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5438.678726] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing set_default_debug -1 all [ 5440.793855] Lustre: DEBUG MARKER: oleg208-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 5443.552868] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5443.711303] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5445.243844] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing set_default_debug -1 all [ 5448.287185] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5448.406717] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5450.894134] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing set_default_debug -1 all [ 5452.670442] Lustre: DEBUG MARKER: oleg208-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 5453.541122] LustreError: 313462:0:(ldlm_lib.c:1113:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5453.548084] LustreError: 313462:0:(ldlm_lib.c:1113:target_handle_connect()) Skipped 2 previous similar messages [ 5455.122883] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5455.185382] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 5456.972479] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing set_default_debug -1 all [ 5459.307242] Lustre: DEBUG MARKER: oleg208-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid [ 5460.966901] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:33) [ 5460.967974] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:65) [ 5466.140018] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing wait_import_state (FULL|IDLE) os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [ 5466.206363] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 5546.976698] LustreError: lustre-OST0001-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 5546.981179] Lustre: lustre-OST0001-osc-MDT0001: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5546.989194] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 5548.004523] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 5548.014095] LustreError: lustre-OST0001-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 5548.014127] Lustre: lustre-OST0001-osc-MDT0000: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5549.747184] LustreError: 315309:0:(obd_class.h:479:obd_check_dev()) Device 27 not setup [ 5549.772879] Lustre: server umount lustre-OST0001 complete [ 5551.013259] LustreError: 314115:0:(ldlm_lib.c:1113:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5553.120175] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 5553.122159] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5553.125572] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5553.127152] Lustre: Skipped 1 previous similar message [ 5556.195604] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 5556.198436] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5556.198568] LustreError: 313461:0:(ldlm_lib.c:1113:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5556.198592] LustreError: 313461:0:(ldlm_lib.c:1113:target_handle_connect()) Skipped 1 previous similar message [ 5556.199961] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5557.867086] LustreError: 315511:0:(obd_class.h:479:obd_check_dev()) Device 23 not setup [ 5557.868619] LustreError: 315511:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 5557.924462] Lustre: server umount lustre-OST0000 complete [ 5561.313071] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 5561.313430] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5561.314620] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5561.314626] Lustre: Skipped 1 previous similar message [ 5561.323959] Lustre: Skipped 1 previous similar message [ 5565.503369] LustreError: 315712:0:(obd_class.h:479:obd_check_dev()) Device 10 not setup [ 5565.505021] LustreError: 315712:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 5565.533692] Lustre: server umount lustre-MDT0000 complete [ 5566.432343] LustreError: 311921:0:(ldlm_lib.c:1113:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5566.437907] LustreError: 311921:0:(ldlm_lib.c:1113:target_handle_connect()) Skipped 2 previous similar messages [ 5566.763483] LustreError: 311902:0:(ldlm_lockd.c:2550:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1755044931 with bad export cookie 16834807297718266549 [ 5566.771375] LustreError: MGC192.168.202.108@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5566.772103] LustreError: 311902:0:(ldlm_lockd.c:2550:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 5566.940175] Lustre: server umount lustre-MDT0001 complete [ 5572.221520] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing unload_modules_local [ 5573.354868] Key type lgssc unregistered [ 5573.515631] LNet: 316440:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5574.565440] LNet: Removed LNI 192.168.202.108@tcp [ 5574.858645] Key type .llcrypt unregistered [ 5574.859574] Key type ._llcrypt unregistered [ 5591.878741] Key type ._llcrypt registered [ 5591.879679] Key type .llcrypt registered [ 5591.910673] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing load_modules_local [ 5592.321617] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5592.328328] alg: No test for adler32 (adler32-zlib) [ 5593.171656] Lustre: Lustre: Build Version: 2.16.56_30_ga643f08 [ 5593.246936] LNet: Added LNI 192.168.202.108@tcp [8/256/0/180] [ 5593.248393] LNet: Accept secure, port 988 [ 5594.823149] Key type lgssc registered [ 5595.158160] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5598.037830] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5598.043757] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5599.209280] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5600.396841] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing set_default_debug -1 all [ 5603.081492] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5603.200091] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5604.436187] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing set_default_debug -1 all [ 5605.392698] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 5607.552269] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5607.685761] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5609.466355] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing set_default_debug -1 all [ 5609.765129] LustreError: 320119:0:(ldlm_lib.c:1113:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5609.768064] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:97) [ 5609.769712] LustreError: 320119:0:(ldlm_lib.c:1113:target_handle_connect()) Skipped 2 previous similar messages [ 5612.026331] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5612.093122] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 5613.161522] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:65) [ 5613.717040] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing set_default_debug -1 all [ 5617.974229] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 5621.956564] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 5634.022510] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5634.023877] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 5634.025396] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5634.025706] Lustre: Skipped 2 previous similar messages [ 5636.354046] LustreError: 322058:0:(obd_class.h:479:obd_check_dev()) Device 10 not setup [ 5636.383906] Lustre: server umount lustre-MDT0000 complete [ 5637.612365] LustreError: 318656:0:(ldlm_lockd.c:2550:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1755045002 with bad export cookie 7915042685038159284 [ 5637.617711] LustreError: MGC192.168.202.108@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5637.693853] LustreError: 322260:0:(obd_class.h:479:obd_check_dev()) Device 18 not setup [ 5637.696873] LustreError: 322260:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 5637.788147] Lustre: server umount lustre-MDT0001 complete [ 5649.332341] LustreError: 322460:0:(obd_class.h:479:obd_check_dev()) Device 23 not setup [ 5649.334324] LustreError: 322460:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 5649.349693] Lustre: server umount lustre-OST0000 complete [ 5660.914065] LustreError: 322661:0:(obd_class.h:479:obd_check_dev()) Device 2 not setup [ 5660.915982] LustreError: 322661:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 5661.014639] Lustre: server umount lustre-OST0001 complete [ 5667.762447] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing load_modules_local [ 5671.790159] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5671.984893] LustreError: 323827:0:(ldlm_lib.c:1113:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5672.012361] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5673.663238] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing set_default_debug -1 all [ 5676.717493] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5678.651923] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing set_default_debug -1 all [ 5679.625088] Lustre: 324925:0:(mgs_llog.c:1345:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 5681.940684] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5682.020246] LustreError: 325279:0:(ldlm_lib.c:1113:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5682.025553] LustreError: 325279:0:(ldlm_lib.c:1113:target_handle_connect()) Skipped 1 previous similar message [ 5682.099558] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5682.101774] Lustre: Skipped 1 previous similar message [ 5682.688753] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:129) [ 5684.456753] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing set_default_debug -1 all [ 5686.820858] LustreError: 325279:0:(ldlm_lib.c:1113:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5686.832239] LustreError: 325279:0:(ldlm_lib.c:1113:target_handle_connect()) Skipped 2 previous similar messages [ 5687.472268] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5689.078623] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:97) [ 5689.716501] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing set_default_debug -1 all [ 5693.388886] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 5694.709692] Lustre: 326758:0:(mgs_llog.c:1345:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 5702.624422] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 5702.628937] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5702.637447] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5702.639258] Lustre: Skipped 3 previous similar messages [ 5706.954186] LustreError: 326861:0:(obd_class.h:479:obd_check_dev()) Device 10 not setup [ 5706.957123] LustreError: 326861:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 5707.000764] Lustre: server umount lustre-MDT0000 complete [ 5708.258614] LustreError: 323827:0:(ldlm_lib.c:1113:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5708.263224] LustreError: 323827:0:(ldlm_lib.c:1113:target_handle_connect()) Skipped 2 previous similar messages [ 5708.402647] LustreError: 324549:0:(ldlm_lockd.c:2550:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1755045073 with bad export cookie 7915042685038160943 [ 5708.404896] LustreError: MGC192.168.202.108@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5708.407228] LustreError: 324549:0:(ldlm_lockd.c:2550:ldlm_cancel_handler()) Skipped 5 previous similar messages [ 5708.591658] Lustre: server umount lustre-MDT0001 complete [ 5720.563132] LustreError: 327263:0:(obd_class.h:479:obd_check_dev()) Device 23 not setup [ 5720.564861] LustreError: 327263:0:(obd_class.h:479:obd_check_dev()) Skipped 15 previous similar messages [ 5720.580402] Lustre: server umount lustre-OST0000 complete [ 5731.137380] Lustre: server umount lustre-OST0001 complete [ 5732.746373] Lustre: DEBUG MARKER: SKIP: conf-sanity test_114 skipping SLOW test 114 [ 5733.498718] Lustre: DEBUG MARKER: SKIP: conf-sanity test_115 skipping excluded test 115 [ 5734.297586] Lustre: DEBUG MARKER: == conf-sanity test 116: big size MDT support ============ 20:31:39 (1755045099) [ 5742.638582] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing load_modules_local [ 5745.566887] loop0: detected capacity change from 0 to 1099511627776 [ 5745.756485] SGI XFS with ACLs, security attributes, quota, no debug enabled [ 5745.758806] *** VALIDATE xfs *** [ 5745.765578] XFS (loop0): Mounting V5 Filesystem [ 5745.773912] XFS (loop0): Ending clean mount [ 5746.230010] loop1: detected capacity change from 0 to 18691697672192 [ 5760.091612] LDISKFS-fs (loop1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5766.668832] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing set_hostid [ 5770.138046] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing load_modules_local [ 5774.490878] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5777.731622] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5780.540737] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5783.453828] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5786.587279] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5786.626416] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5786.766600] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 5786.785379] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 5786.827043] Lustre: lustre-MDT0000: new disk, initializing [ 5786.858602] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5786.860552] Lustre: Skipped 1 previous similar message [ 5786.865961] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 5788.572455] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing set_default_debug -1 all [ 5793.961303] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5794.002412] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5794.044242] Lustre: 331807:0:(mgs_llog.c:1345:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 5794.083591] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 5794.086183] Lustre: Skipped 1 previous similar message [ 5794.125537] Lustre: lustre-MDT0001: new disk, initializing [ 5794.187859] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 5794.192210] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 5795.903916] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing set_default_debug -1 all [ 5800.059558] Lustre: DEBUG MARKER: oleg208-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 5801.645378] Lustre: DEBUG MARKER: oleg208-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 5804.606479] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5804.647660] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5804.815311] Lustre: lustre-OST0000: new disk, initializing [ 5804.819441] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 5806.223806] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 5806.228263] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 5806.283638] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 5807.473792] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing set_default_debug -1 all [ 5811.307642] Lustre: DEBUG MARKER: oleg208-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 5813.094616] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 5813.193642] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 5815.042193] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 5815.155716] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 5816.804247] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 5816.809469] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5816.813600] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5816.817524] Lustre: Skipped 4 previous similar messages [ 5816.820924] Lustre: Skipped 4 previous similar messages [ 5821.924840] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5821.927637] Lustre: Skipped 2 previous similar messages [ 5822.190105] LustreError: 334714:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 5822.191807] LustreError: 334714:0:(obd_class.h:479:obd_check_dev()) Skipped 9 previous similar messages [ 5822.255467] Lustre: server umount lustre-OST0000 complete [ 5824.992143] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 5824.994455] LustreError: Skipped 1 previous similar message [ 5824.995606] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5827.040833] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5827.043628] Lustre: Skipped 2 previous similar messages [ 5829.599537] Lustre: server umount lustre-MDT0000 complete [ 5830.801646] LustreError: 332580:0:(ldlm_lockd.c:2550:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1755045195 with bad export cookie 7915042685038162497 [ 5830.805304] LustreError: MGC192.168.202.108@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5830.953653] Lustre: server umount lustre-MDT0001 complete [ 5833.861850] XFS (loop0): Unmounting Filesystem [ 5835.494961] Lustre: DEBUG MARKER: == conf-sanity test 117: lctl get_param return errors properly ========================================================== 20:33:21 (1755045201) [ 5838.102952] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5838.291453] LustreError: 335935:0:(ldlm_lib.c:1113:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5838.326722] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5838.330222] Lustre: Skipped 2 previous similar messages [ 5839.844848] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing set_default_debug -1 all [ 5843.074593] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5844.661307] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing set_default_debug -1 all [ 5846.655682] Lustre: DEBUG MARKER: oleg208-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 5847.822574] Lustre: DEBUG MARKER: oleg208-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 5849.916791] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5851.686600] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing set_default_debug -1 all [ 5853.306492] Lustre: DEBUG MARKER: oleg208-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 5868.513931] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5868.514801] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 5868.518670] Lustre: Skipped 1 previous similar message [ 5868.522620] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5868.524056] Lustre: Skipped 1 previous similar message [ 5872.940993] LustreError: 338279:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 5872.942566] LustreError: 338279:0:(obd_class.h:479:obd_check_dev()) Skipped 17 previous similar messages [ 5872.997842] Lustre: server umount lustre-OST0000 complete [ 5878.752780] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5878.757931] Lustre: Skipped 2 previous similar messages [ 5880.278633] Lustre: server umount lustre-MDT0000 complete [ 5881.394209] LustreError: 335916:0:(ldlm_lockd.c:2550:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1755045246 with bad export cookie 7915042685038163379 [ 5881.396796] LustreError: 335916:0:(ldlm_lockd.c:2550:ldlm_cancel_handler()) Skipped 5 previous similar messages [ 5881.398071] LustreError: MGC192.168.202.108@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5881.543866] Lustre: server umount lustre-MDT0001 complete [ 5885.676616] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing unload_modules_local [ 5886.583286] Key type lgssc unregistered [ 5886.712363] LNet: 339211:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5887.781359] LNet: Removed LNI 192.168.202.108@tcp [ 5888.051216] Key type .llcrypt unregistered [ 5888.052213] Key type ._llcrypt unregistered [ 5897.859294] Key type ._llcrypt registered [ 5897.860199] Key type .llcrypt registered [ 5897.890424] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing load_modules_local [ 5898.270040] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5898.286736] alg: No test for adler32 (adler32-zlib) [ 5899.124477] Lustre: Lustre: Build Version: 2.16.56_30_ga643f08 [ 5899.196111] LNet: Added LNI 192.168.202.108@tcp [8/256/0/180] [ 5899.197984] LNet: Accept secure, port 988 [ 5900.775125] Key type lgssc registered [ 5901.084878] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5903.440210] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5903.446620] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5904.592993] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5905.780334] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing set_default_debug -1 all [ 5908.350771] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5908.456719] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5909.605985] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing set_default_debug -1 all [ 5911.256389] Lustre: DEBUG MARKER: oleg208-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 5912.421787] Lustre: DEBUG MARKER: oleg208-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 5914.532207] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5914.592586] LustreError: 342309:0:(ldlm_lib.c:1113:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5914.596222] LustreError: 342309:0:(ldlm_lib.c:1113:target_handle_connect()) Skipped 2 previous similar messages [ 5914.635226] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5916.283588] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing set_default_debug -1 all [ 5917.899875] Lustre: DEBUG MARKER: oleg208-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 5928.928364] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [ 5928.931503] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5928.936508] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 5929.446584] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 5929.446952] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5932.927321] LustreError: 342998:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [ 5932.952923] Lustre: server umount lustre-MDT0001 complete [ 5933.864866] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5934.563846] LustreError: 342862:0:(ldlm_lib.c:1113:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5934.568159] LustreError: 342862:0:(ldlm_lib.c:1113:target_handle_connect()) Skipped 1 previous similar message [ 5936.372451] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5936.404510] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [ 5936.406677] Lustre: Found index 1 for lustre-MDT0001, updating log [ 5936.418693] Lustre: 340715:0:(mgs_llog.c:1345:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 5936.544125] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5937.766549] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing set_default_debug -1 all [ 5941.736154] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to (at 0@lo) [ 5945.316326] LustreError: 344174:0:(obd_mount.c:188:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [ 5945.318435] LustreError: 344174:0:(tgt_mount.c:669:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [ 5945.320531] LustreError: 340717:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [ 5945.334855] LustreError: 344177:0:(obd_config.c:1993:class_config_llog_handler()) MGC192.168.202.108@tcp: cfg command failed: rc = -17 [ 5945.336946] Lustre: cmd=cf001 0:lustre-MDT0001-osp-MDT0000 1:osp 2:lustre-MDT0000-mdtlov_UUID [ 5945.336946] [ 6241.854801] Lustre: lustre-MDT0001: Not available for connect from 192.168.202.8@tcp (stopping) [ 6241.856633] Lustre: Skipped 2 previous similar messages [ 6243.809927] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [ 6243.812058] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6245.433904] LustreError: 355564:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [ 6245.435550] LustreError: 355564:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 6245.458837] Lustre: server umount lustre-MDT0001 complete [ 6246.320282] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6246.969361] LustreError: 340720:0:(ldlm_lib.c:1113:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.202.8@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6246.973371] LustreError: 340720:0:(ldlm_lib.c:1113:target_handle_connect()) Skipped 1 previous similar message [ 6248.746285] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 6248.775464] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [ 6248.776899] Lustre: Found index 1 for lustre-MDT0001, updating log [ 6248.789226] Lustre: 340715:0:(mgs_llog.c:1345:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 6248.918253] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 6250.146222] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing set_default_debug -1 all [ 6254.056171] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to (at 0@lo) [ 6259.181421] LustreError: 356835:0:(obd_mount.c:188:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [ 6259.184706] LustreError: 356835:0:(tgt_mount.c:669:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [ 6259.188149] LustreError: 340717:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [ 6259.190492] LustreError: 340717:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [ 6259.204401] LustreError: 356838:0:(obd_config.c:685:class_setup()) Device 8 already setup (type osp) [ 6259.207162] LustreError: 356838:0:(obd_config.c:1993:class_config_llog_handler()) MGC192.168.202.108@tcp: cfg command failed: rc = -17 [ 6259.210761] Lustre: cmd=cf003 0:lustre-MDT0001-osp-MDT0000 1:lustre-MDT0001_UUID 2:192.168.202.108@tcp [ 6259.210761] [ 6554.174171] Lustre: lustre-MDT0001: Not available for connect from 192.168.202.8@tcp (stopping) [ 6554.177527] Lustre: Skipped 2 previous similar messages [ 6556.129395] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [ 6556.130252] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6556.134069] Lustre: Skipped 1 previous similar message [ 6559.288776] Lustre: lustre-MDT0001: Not available for connect from 192.168.202.8@tcp (stopping) [ 6559.290868] Lustre: Skipped 2 previous similar messages [ 6559.481950] LustreError: 368474:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [ 6559.484318] LustreError: 368474:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 6559.510451] Lustre: server umount lustre-MDT0001 complete [ 6560.356063] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6561.248945] LustreError: 341430:0:(ldlm_lib.c:1113:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6561.254804] LustreError: 341430:0:(ldlm_lib.c:1113:target_handle_connect()) Skipped 1 previous similar message [ 6562.689738] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 6562.716336] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [ 6562.717777] Lustre: Found index 1 for lustre-MDT0001, updating log [ 6562.728937] Lustre: 340713:0:(mgs_llog.c:1345:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 6562.829575] Lustre: lustre-MDT0001: Imperative Recovery enabled, recovery window shrunk from 60-180 down to 60-180 [ 6564.019559] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing set_default_debug -1 all [ 6567.919285] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to (at 0@lo) [ 6568.427426] LustreError: 369543:0:(obd_mount.c:188:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [ 6568.432085] LustreError: 369543:0:(tgt_mount.c:669:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [ 6568.437285] LustreError: 340717:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [ 6568.441306] LustreError: 340717:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [ 6568.473975] LustreError: 369546:0:(lod_lov.c:207:lod_add_device()) lustre-MDT0001-osp-MDT0000: cannot connect to next dev lustre-MDT0001_UUID (-114) [ 6568.483720] LustreError: 369546:0:(obd_config.c:1993:class_config_llog_handler()) MGC192.168.202.108@tcp: cfg command failed: rc = -114 [ 6568.489301] Lustre: cmd=cf014 0:lustre-MDT0000-mdtlov 1:lustre-MDT0001_UUID 2:1 3:1 [ 6568.489301] [ 6880.226407] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6880.226426] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 6880.227687] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6880.228975] Lustre: Skipped 3 previous similar messages [ 6883.962238] LustreError: 381551:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup [ 6883.963734] LustreError: 381551:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 6883.987706] Lustre: server umount lustre-MDT0000 complete [ 6885.044095] LustreError: 340707:0:(ldlm_lockd.c:2550:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1755046250 with bad export cookie 6788670514259428810 [ 6885.047014] LustreError: 340707:0:(ldlm_lockd.c:2550:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 6885.047668] LustreError: MGC192.168.202.108@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6885.343793] LustreError: 342862:0:(ldlm_lib.c:1113:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6891.193660] LustreError: 381752:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [ 6891.195199] LustreError: 381752:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 6891.265773] Lustre: server umount lustre-MDT0001 complete [ 6892.493458] Lustre: server umount lustre-OST0000 complete [ 6895.032781] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing set_hostid [ 6898.145295] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing load_modules_local [ 6901.114923] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6903.031611] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6904.767190] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6906.614687] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6908.615548] Lustre: DEBUG MARKER: == conf-sanity test 120: cross-target rename should not create bad symlinks ========================================================== 20:51:14 (1755046274) [ 6911.068451] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6911.086081] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6911.158444] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 6911.167560] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 6911.201792] Lustre: lustre-MDT0000: new disk, initializing [ 6911.223495] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6911.228820] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 6912.589501] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing set_default_debug -1 all [ 6916.951582] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6916.974187] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 6917.005947] Lustre: 384467:0:(mgs_llog.c:1345:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 6917.028355] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 6917.029939] Lustre: Skipped 1 previous similar message [ 6917.065139] Lustre: lustre-MDT0001: new disk, initializing [ 6917.099097] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 6917.112456] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 6917.117143] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 6918.434851] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing set_default_debug -1 all [ 6921.630886] Lustre: DEBUG MARKER: oleg208-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 6922.885909] Lustre: DEBUG MARKER: oleg208-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 6925.282200] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6925.311727] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 6925.436703] Lustre: lustre-OST0000: new disk, initializing [ 6925.439711] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 6925.482317] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 6927.440680] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing set_default_debug -1 all [ 6929.274960] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 6929.279190] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 6929.311744] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 6930.620337] Lustre: DEBUG MARKER: oleg208-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 6934.498529] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6934.500111] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6934.501913] Lustre: Skipped 2 previous similar messages [ 6934.506053] Lustre: Skipped 6 previous similar messages [ 6939.651528] LustreError: 386892:0:(obd_class.h:479:obd_check_dev()) Device 22 not setup [ 6939.652962] LustreError: 386892:0:(obd_class.h:479:obd_check_dev()) Skipped 11 previous similar messages [ 6939.687974] Lustre: server umount lustre-MDT0000 complete [ 6941.191916] LustreError: 386201:0:(ldlm_lockd.c:2550:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1755046306 with bad export cookie 6788670514259430763 [ 6941.196846] LustreError: 385254:0:(osp_object.c:617:osp_attr_get()) lustre-MDT0000-osp-MDT0001: osp_attr_get update error [0x200000401:0x2:0x0]: rc = -5 [ 6941.198257] LustreError: MGC192.168.202.108@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6941.198347] LustreError: 386201:0:(ldlm_lockd.c:2550:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 6941.413509] Lustre: server umount lustre-MDT0001 complete [ 6953.240544] Lustre: server umount lustre-OST0000 complete [ 6966.081338] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing set_hostid [ 6970.743096] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing load_modules_local [ 6975.658850] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6979.302155] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6982.275484] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6985.376818] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6986.122328] Lustre: DEBUG MARKER: == conf-sanity test complete, duration 6844 sec ========== 20:52:31 (1755046351) [ 6986.836718] Lustre: DEBUG MARKER: === conf-sanity: start cleanup 20:52:32 (1755046352) === [ 6988.134924] Lustre: DEBUG MARKER: === conf-sanity: finish cleanup 20:52:33 (1755046353) === [ 7007.273704] Lustre: DEBUG MARKER: oleg208-server.virtnet: executing unload_modules_local [ 7008.781225] Key type lgssc unregistered [ 7008.963438] LNet: 392377:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 7010.021575] LNet: Removed LNI 192.168.202.108@tcp [ 7010.510187] Key type .llcrypt unregistered [ 7010.512103] Key type ._llcrypt unregistered