[ 0.000000] Linux version 4.18.0rh8.10-debug (green@maintenance) (gcc version 8.5.0 20210514 (Red Hat 8.5.0-26) (GCC)) #2 SMP Mon Jul 14 01:24:22 EDT 2025 [ 0.000000] Command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers' [ 0.000000] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256 [ 0.000000] x86/fpu: Enabled xstate features 0x7, context size is 832 bytes, using 'standard' format. [ 0.000000] signal: max sigframe size: 1776 [ 0.000000] BIOS-provided physical RAM map: [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x00000000bffcdfff] usable [ 0.000000] BIOS-e820: [mem 0x00000000bffce000-0x00000000bfffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x0000000146dfffff] usable [ 0.000000] NX (Execute Disable) protection: active [ 0.000000] SMBIOS 2.8 present. [ 0.000000] DMI: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.16.2-1.fc38 04/01/2014 [ 0.000000] Hypervisor detected: KVM [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00 [ 0.000000] kvm-clock: using sched offset of 523485524 cycles [ 0.000000] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns [ 0.000000] tsc: Detected 2399.998 MHz processor [ 0.000000] last_pfn = 0x146e00 max_arch_pfn = 0x400000000 [ 0.000000] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT [ 0.000000] last_pfn = 0xbffce max_arch_pfn = 0x400000000 [ 0.000000] found SMP MP-table at [mem 0x000f5b30-0x000f5b3f] [ 0.000000] RAMDISK: [mem 0xbcc54000-0xbffbffff] [ 0.000000] ACPI: Early table checksum verification disabled [ 0.000000] ACPI: RSDP 0x00000000000F5950 000014 (v00 BOCHS ) [ 0.000000] ACPI: RSDT 0x00000000BFFE1BB7 000034 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACP 0x00000000BFFE1A53 000074 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: DSDT 0x00000000BFFE0040 001A13 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACS 0x00000000BFFE0000 000040 [ 0.000000] ACPI: APIC 0x00000000BFFE1AC7 000090 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: HPET 0x00000000BFFE1B57 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: WAET 0x00000000BFFE1B8F 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: Reserving FACP table memory at [mem 0xbffe1a53-0xbffe1ac6] [ 0.000000] ACPI: Reserving DSDT table memory at [mem 0xbffe0040-0xbffe1a52] [ 0.000000] ACPI: Reserving FACS table memory at [mem 0xbffe0000-0xbffe003f] [ 0.000000] ACPI: Reserving APIC table memory at [mem 0xbffe1ac7-0xbffe1b56] [ 0.000000] ACPI: Reserving HPET table memory at [mem 0xbffe1b57-0xbffe1b8e] [ 0.000000] ACPI: Reserving WAET table memory at [mem 0xbffe1b8f-0xbffe1bb6] [ 0.000000] No NUMA configuration found [ 0.000000] Faking a node at [mem 0x0000000000000000-0x0000000146dfffff] [ 0.000000] NODE_DATA(0) allocated [mem 0x1465a3000-0x1465cdfff] [ 0.000000] Reserving 256MB of memory at 2752MB for crashkernel (System RAM: 4205MB) [ 0.000000] Zone ranges: [ 0.000000] DMA [mem 0x0000000000001000-0x0000000000ffffff] [ 0.000000] DMA32 [mem 0x0000000001000000-0x00000000ffffffff] [ 0.000000] Normal [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Device empty [ 0.000000] Movable zone start for each node [ 0.000000] Early memory node ranges [ 0.000000] node 0: [mem 0x0000000000001000-0x000000000009efff] [ 0.000000] node 0: [mem 0x0000000000100000-0x00000000bffcdfff] [ 0.000000] node 0: [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Zeroed struct page in unavailable ranges: 4756 pages [ 0.000000] Initmem setup node 0 [mem 0x0000000000001000-0x0000000146dfffff] [ 0.000000] ACPI: PM-Timer IO Port: 0x608 [ 0.000000] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1]) [ 0.000000] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23 [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level) [ 0.000000] Using ACPI (MADT) for SMP configuration information [ 0.000000] ACPI: HPET id: 0x8086a201 base: 0xfed00000 [ 0.000000] TSC deadline timer available [ 0.000000] smpboot: Allowing 4 CPUs, 0 hotplug CPUs [ 0.000000] kvm-guest: KVM setup pv remote TLB flush [ 0.000000] kvm-guest: setup PV sched yield [ 0.000000] PM: Registered nosave memory: [mem 0x00000000-0x00000fff] [ 0.000000] PM: Registered nosave memory: [mem 0x0009f000-0x0009ffff] [ 0.000000] PM: Registered nosave memory: [mem 0x000a0000-0x000effff] [ 0.000000] PM: Registered nosave memory: [mem 0x000f0000-0x000fffff] [ 0.000000] PM: Registered nosave memory: [mem 0xbffce000-0xbfffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xc0000000-0xfeffbfff] [ 0.000000] PM: Registered nosave memory: [mem 0xfeffc000-0xfeffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xff000000-0xfffbffff] [ 0.000000] PM: Registered nosave memory: [mem 0xfffc0000-0xffffffff] [ 0.000000] [mem 0xc0000000-0xfeffbfff] available for PCI devices [ 0.000000] Booting paravirtualized kernel on KVM [ 0.000000] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns [ 0.000000] setup_percpu: NR_CPUS:8192 nr_cpumask_bits:4 nr_cpu_ids:4 nr_node_ids:1 [ 0.000000] percpu: Embedded 63 pages/cpu s221184 r8192 d28672 u524288 [ 0.000000] kvm-guest: PV spinlocks enabled [ 0.000000] PV qspinlock hash table entries: 256 (order: 0, 4096 bytes, linear) [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 1059606 [ 0.000000] Policy zone: Normal [ 0.000000] Kernel command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] Specific versions of hardware are certified with Red Hat Enterprise Linux 8. Please see the list of hardware certified with Red Hat Enterprise Linux 8 at https://catalog.redhat.com. [ 0.000000] audit: disabled (until reboot) [ 0.000000] software IO TLB: area num 4. [ 0.000000] Memory: 2895240K/4306352K available (18435K kernel code, 11221K rwdata, 7248K rodata, 2908K init, 18040K bss, 524580K reserved, 0K cma-reserved) [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1 [ 0.000000] kmemleak: Kernel memory leak detector disabled [ 0.000000] ftrace: allocating 41240 entries in 162 pages [ 0.000000] ftrace: allocated 162 pages with 3 groups [ 0.000000] rcu: Hierarchical RCU implementation. [ 0.000000] rcu: RCU event tracing is enabled. [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=8192 to nr_cpu_ids=4. [ 0.000000] rcu: RCU callback double-/use-after-free debug enabled. [ 0.000000] Rude variant of Tasks RCU enabled. [ 0.000000] Tracing variant of Tasks RCU enabled. [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies. [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4 [ 0.000000] NR_IRQS: 524544, nr_irqs: 456, preallocated irqs: 16 [ 0.000000] random: get_random_bytes called from start_kernel+0x622/0x9a8 with crng_init=0 [ 0.001000] Console: colour *CGA 80x25 [ 0.001000] printk: console [ttyS1] enabled [ 0.001000] ACPI: Core revision 20220331 [ 0.001000] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns [ 0.001010] APIC: Switch to symmetric I/O mode setup [ 0.003267] x2apic enabled [ 0.004017] Switched APIC routing to physical x2apic. [ 0.005000] kvm-guest: setup PV IPIs [ 0.007514] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1 [ 0.008000] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 0.008029] Calibrating delay loop (skipped) preset value.. 4799.99 BogoMIPS (lpj=2399998) [ 0.009020] pid_max: default: 32768 minimum: 301 [ 0.010144] LSM: Security Framework initializing [ 0.011034] Yama: becoming mindful. [ 0.012024] SELinux: Initializing. [ 0.013045] *** VALIDATE selinux *** [ 0.018708] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, vmalloc) [ 0.021701] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, vmalloc) [ 0.022134] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.023084] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.024095] *** VALIDATE tmpfs *** [ 0.025386] *** VALIDATE proc *** [ 0.027181] *** VALIDATE cgroup *** [ 0.028005] *** VALIDATE cgroup2 *** [ 0.029218] x86/cpu: User Mode Instruction Prevention (UMIP) activated [ 0.030122] Last level iTLB entries: 4KB 0, 2MB 0, 4MB 0 [ 0.031003] Last level dTLB entries: 4KB 0, 2MB 0, 4MB 0, 1GB 0 [ 0.032021] Spectre V2 : User space: Vulnerable [ 0.033000] Speculative Store Bypass: Vulnerable [ 0.034936] debug: unmapping init [mem 0xffffffffa6059000-0xffffffffa6060fff] [ 0.036139] smpboot: CPU0: Intel(R) Xeon(R) CPU E5-2695 v2 @ 2.40GHz (family: 0x6, model: 0x3e, stepping: 0x4) [ 0.037571] Performance Events: IvyBridge events, full-width counters, Intel PMU driver. [ 0.038013] ... version: 2 [ 0.039007] ... bit width: 48 [ 0.040006] ... generic registers: 4 [ 0.041007] ... value mask: 0000ffffffffffff [ 0.042009] ... max period: 00007fffffffffff [ 0.043008] ... fixed-purpose events: 3 [ 0.044008] ... event mask: 000000070000000f [ 0.045452] rcu: Hierarchical SRCU implementation. [ 0.049536] smp: Bringing up secondary CPUs ... [ 0.050528] x86: Booting SMP configuration: [ 0.051018] .... node #0, CPUs: #1 #2 #3 [ 0.056330] smp: Brought up 1 node, 4 CPUs [ 0.058012] smpboot: Max logical packages: 1 [ 0.059012] smpboot: Total of 4 processors activated (19199.98 BogoMIPS) [ 0.098000] node 0 deferred pages initialised in 35ms [ 0.101271] devtmpfs: initialized [ 0.102000] x86/mm: Memory block size: 128MB [ 0.104283] gcov: version magic: 0x41383552 [ 0.107378] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns [ 0.108128] futex hash table entries: 1024 (order: 4, 65536 bytes, vmalloc) [ 0.109577] pinctrl core: initialized pinctrl subsystem [ 0.111045] [ 0.111919] ************************************************************* [ 0.112017] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.113017] ** ** [ 0.114175] ** IOMMU DebugFS SUPPORT HAS BEEN ENABLED IN THIS KERNEL ** [ 0.115013] ** ** [ 0.116019] ** This means that this kernel is built to expose internal ** [ 0.117018] ** IOMMU data structures, which may compromise security on ** [ 0.118013] ** your system. ** [ 0.119013] ** ** [ 0.120011] ** If you see this message and you are not debugging the ** [ 0.121015] ** kernel, report this immediately to your vendor! ** [ 0.122011] ** ** [ 0.123013] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.124010] ************************************************************* [ 0.125903] NET: Registered protocol family 16 [ 0.126448] DMA: preallocated 512 KiB GFP_KERNEL pool for atomic allocations [ 0.127081] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations [ 0.128145] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations [ 0.129901] cpuidle: using governor menu [ 0.131000] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5 [ 0.134708] PCI: Using configuration type 1 for base access [ 0.137157] core: PMU erratum BJ122, BV98, HSD29 worked around, HT is on [ 0.148649] HugeTLB registered 1.00 GiB page size, pre-allocated 0 pages [ 0.149020] HugeTLB registered 2.00 MiB page size, pre-allocated 0 pages [ 0.152038] cryptd: max_cpu_qlen set to 1000 [ 0.155633] ACPI: Added _OSI(Module Device) [ 0.156016] ACPI: Added _OSI(Processor Device) [ 0.157011] ACPI: Added _OSI(3.0 _SCP Extensions) [ 0.158013] ACPI: Added _OSI(Processor Aggregator Device) [ 0.162281] ACPI: 1 ACPI AML tables successfully acquired and loaded [ 0.166669] ACPI: Interpreter enabled [ 0.167055] ACPI: PM: (supports S0 S3 S4 S5) [ 0.168014] ACPI: Using IOAPIC for interrupt routing [ 0.170249] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug [ 0.174447] ACPI: Enabled 2 GPEs in block 00 to 0F [ 0.185236] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff]) [ 0.187038] acpi PNP0A03:00: _OSC: OS supports [ASPM ClockPM Segments MSI HPX-Type3] [ 0.189021] acpi PNP0A03:00: _OSC: not requesting OS control; OS requires [ExtendedConfig ASPM ClockPM MSI] [ 0.192076] acpi PNP0A03:00: fail to add MMCONFIG information, can't access extended PCI configuration space under this bridge. [ 0.196235] acpiphp: Slot [2] registered [ 0.197074] acpiphp: Slot [3] registered [ 0.198125] acpiphp: Slot [4] registered [ 0.200077] acpiphp: Slot [5] registered [ 0.201083] acpiphp: Slot [6] registered [ 0.202092] acpiphp: Slot [7] registered [ 0.203063] acpiphp: Slot [8] registered [ 0.204063] acpiphp: Slot [9] registered [ 0.205069] acpiphp: Slot [10] registered [ 0.206063] acpiphp: Slot [11] registered [ 0.207059] acpiphp: Slot [12] registered [ 0.208042] acpiphp: Slot [13] registered [ 0.210439] acpiphp: Slot [14] registered [ 0.214857] acpiphp: Slot [15] registered [ 0.216130] acpiphp: Slot [16] registered [ 0.218172] acpiphp: Slot [17] registered [ 0.221171] acpiphp: Slot [18] registered [ 0.223151] acpiphp: Slot [19] registered [ 0.225161] acpiphp: Slot [20] registered [ 0.226000] acpiphp: Slot [21] registered [ 0.228188] acpiphp: Slot [22] registered [ 0.230076] acpiphp: Slot [23] registered [ 0.231080] acpiphp: Slot [24] registered [ 0.232645] acpiphp: Slot [25] registered [ 0.233114] acpiphp: Slot [26] registered [ 0.234113] acpiphp: Slot [27] registered [ 0.236088] acpiphp: Slot [28] registered [ 0.237070] acpiphp: Slot [29] registered [ 0.238000] acpiphp: Slot [30] registered [ 0.238070] acpiphp: Slot [31] registered [ 0.239000] PCI host bridge to bus 0000:00 [ 0.239000] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window] [ 0.244036] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window] [ 0.246020] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window] [ 0.252031] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window] [ 0.257037] pci_bus 0000:00: root bus resource [mem 0x180000000-0x1ffffffff window] [ 0.260031] pci_bus 0000:00: root bus resource [bus 00-ff] [ 0.263233] pci 0000:00:00.0: [8086:1237] type 00 class 0x060000 [ 0.267000] pci 0000:00:01.0: [8086:7000] type 00 class 0x060100 [ 0.274130] pci 0000:00:01.1: [8086:7010] type 00 class 0x010180 [ 0.285734] pci 0000:00:01.1: reg 0x20: [io 0xc320-0xc32f] [ 0.289000] pci 0000:00:01.1: legacy IDE quirk: reg 0x10: [io 0x01f0-0x01f7] [ 0.289000] pci 0000:00:01.1: legacy IDE quirk: reg 0x14: [io 0x03f6] [ 0.289000] pci 0000:00:01.1: legacy IDE quirk: reg 0x18: [io 0x0170-0x0177] [ 0.289000] pci 0000:00:01.1: legacy IDE quirk: reg 0x1c: [io 0x0376] [ 0.289000] pci 0000:00:01.3: [8086:7113] type 00 class 0x068000 [ 0.292767] pci 0000:00:01.3: quirk: [io 0x0600-0x063f] claimed by PIIX4 ACPI [ 0.294052] pci 0000:00:01.3: quirk: [io 0x0700-0x070f] claimed by PIIX4 SMB [ 0.295000] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 [ 0.297000] pci 0000:00:02.0: reg 0x10: [io 0xc300-0xc31f] [ 0.304020] pci 0000:00:02.0: reg 0x20: [mem 0xfebe4000-0xfebe7fff 64bit pref] [ 0.309201] pci 0000:00:02.0: reg 0x30: [mem 0xfeb80000-0xfebbffff pref] [ 0.313000] pci 0000:00:05.0: [1af4:1001] type 00 class 0x010000 [ 0.319019] pci 0000:00:05.0: reg 0x10: [io 0xc000-0xc07f] [ 0.325019] pci 0000:00:05.0: reg 0x14: [mem 0xfebc0000-0xfebc0fff] [ 0.349024] pci 0000:00:05.0: reg 0x20: [mem 0xfebe8000-0xfebebfff 64bit pref] [ 0.365431] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 [ 0.382024] pci 0000:00:06.0: reg 0x10: [io 0xc080-0xc0ff] [ 0.394039] pci 0000:00:06.0: reg 0x14: [mem 0xfebc1000-0xfebc1fff] [ 0.424033] pci 0000:00:06.0: reg 0x20: [mem 0xfebec000-0xfebeffff 64bit pref] [ 0.440570] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 [ 0.451023] pci 0000:00:07.0: reg 0x10: [io 0xc100-0xc17f] [ 0.467032] pci 0000:00:07.0: reg 0x14: [mem 0xfebc2000-0xfebc2fff] [ 0.495016] pci 0000:00:07.0: reg 0x20: [mem 0xfebf0000-0xfebf3fff 64bit pref] [ 0.508427] pci 0000:00:08.0: [1af4:1001] type 00 class 0x010000 [ 0.524032] pci 0000:00:08.0: reg 0x10: [io 0xc180-0xc1ff] [ 0.531000] pci 0000:00:08.0: reg 0x14: [mem 0xfebc3000-0xfebc3fff] [ 0.545017] pci 0000:00:08.0: reg 0x20: [mem 0xfebf4000-0xfebf7fff 64bit pref] [ 0.577396] pci 0000:00:09.0: [1af4:1001] type 00 class 0x010000 [ 0.593023] pci 0000:00:09.0: reg 0x10: [io 0xc200-0xc27f] [ 0.606050] pci 0000:00:09.0: reg 0x14: [mem 0xfebc4000-0xfebc4fff] [ 0.630023] pci 0000:00:09.0: reg 0x20: [mem 0xfebf8000-0xfebfbfff 64bit pref] [ 0.644548] pci 0000:00:0a.0: [1af4:1001] type 00 class 0x010000 [ 0.660017] pci 0000:00:0a.0: reg 0x10: [io 0xc280-0xc2ff] [ 0.671095] pci 0000:00:0a.0: reg 0x14: [mem 0xfebc5000-0xfebc5fff] [ 0.689077] pci 0000:00:0a.0: reg 0x20: [mem 0xfebfc000-0xfebfffff 64bit pref] [ 0.716035] ACPI: PCI: Interrupt link LNKA configured for IRQ 10 [ 0.723031] ACPI: PCI: Interrupt link LNKB configured for IRQ 10 [ 0.729401] ACPI: PCI: Interrupt link LNKC configured for IRQ 11 [ 0.732343] ACPI: PCI: Interrupt link LNKD configured for IRQ 11 [ 0.734173] ACPI: PCI: Interrupt link LNKS configured for IRQ 9 [ 0.740209] iommu: Default domain type: Passthrough [ 0.742621] SCSI subsystem initialized [ 0.744152] ACPI: bus type USB registered [ 0.746000] usbcore: registered new interface driver usbfs [ 0.746000] usbcore: registered new interface driver hub [ 0.750173] usbcore: registered new device driver usb [ 0.751182] pps_core: LinuxPPS API ver. 1 registered [ 0.753009] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti [ 0.755092] PTP clock support registered [ 0.758024] EDAC MC: Ver: 3.0.0 [ 0.762209] PCI: Using ACPI for IRQ routing [ 0.763734] NetLabel: Initializing [ 0.765088] NetLabel: domain hash size = 128 [ 0.768010] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO [ 0.771169] NetLabel: unlabeled traffic allowed by default [ 0.775405] vgaarb: loaded [ 0.778708] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0 [ 0.781026] hpet0: 3 comparators, 64-bit 100.000000 MHz counter [ 0.790164] clocksource: Switched to clocksource kvm-clock [ 0.943485] VFS: Disk quotas dquot_6.6.0 [ 0.944877] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) [ 0.947427] *** VALIDATE ramfs *** [ 0.948484] *** VALIDATE hugetlbfs *** [ 0.950375] pnp: PnP ACPI init [ 0.952603] pnp: PnP ACPI: found 6 devices [ 0.969128] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns [ 0.973406] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window] [ 0.975266] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window] [ 0.976932] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window] [ 0.979079] pci_bus 0000:00: resource 7 [mem 0xc0000000-0xfebfffff window] [ 0.981207] pci_bus 0000:00: resource 8 [mem 0x180000000-0x1ffffffff window] [ 0.983551] NET: Registered protocol family 2 [ 0.985886] IP idents hash table entries: 131072 (order: 8, 1048576 bytes, vmalloc) [ 0.991465] tcp_listen_portaddr_hash hash table entries: 4096 (order: 5, 163840 bytes, vmalloc) [ 0.996743] TCP established hash table entries: 65536 (order: 7, 524288 bytes, vmalloc) [ 1.002212] TCP bind hash table entries: 65536 (order: 9, 2097152 bytes, vmalloc) [ 1.006809] TCP: Hash tables configured (established 65536 bind 65536) [ 1.012831] MPTCP token hash table entries: 8192 (order: 6, 393216 bytes, vmalloc) [ 1.015783] UDP hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 1.019632] UDP-Lite hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 1.025905] NET: Registered protocol family 1 [ 1.032885] RPC: Registered named UNIX socket transport module. [ 1.035485] RPC: Registered udp transport module. [ 1.037281] RPC: Registered tcp transport module. [ 1.038790] RPC: Registered tcp NFSv4.1 backchannel transport module. [ 1.043880] NET: Registered protocol family 44 [ 1.045386] pci 0000:00:00.0: Limiting direct PCI/PCI transfers [ 1.047258] pci 0000:00:01.0: PIIX3: Enabling Passive Release [ 1.049071] pci 0000:00:01.0: Activating ISA DMA hang workarounds [ 1.051219] PCI: CLS 0 bytes, default 64 [ 1.052686] Unpacking initramfs... [ 3.269578] debug: unmapping init [mem 0xffff9d01bcc54000-0xffff9d01bffbffff] [ 3.274282] PCI-DMA: Using software bounce buffering for IO (SWIOTLB) [ 3.276799] software IO TLB: mapped [mem 0x00000000a8000000-0x00000000ac000000] (64MB) [ 3.280368] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 3.963761] Initialise system trusted keyrings [ 3.965472] Key type blacklist registered [ 3.967569] workingset: timestamp_bits=36 max_order=20 bucket_order=0 [ 3.982433] zbud: loaded [ 3.986512] *** VALIDATE nfs *** [ 3.988026] *** VALIDATE nfs4 *** [ 3.989677] pstore: using deflate compression [ 3.993300] Platform Keyring initialized [ 4.318027] NET: Registered protocol family 38 [ 4.319799] Key type asymmetric registered [ 4.325146] Asymmetric key parser 'x509' registered [ 4.329540] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 247) [ 4.337093] io scheduler mq-deadline registered [ 4.342512] io scheduler kyber registered [ 4.345585] io scheduler bfq registered [ 4.350290] atomic64_test: passed for x86-64 platform with CX8 and with SSE [ 4.356755] shpchp: Standard Hot Plug PCI Controller Driver version: 0.4 [ 4.363604] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0 [ 4.372476] ACPI: Power Button [PWRF] [ 4.669393] ACPI: \_SB_.LNKB: Enabled at IRQ 10 [ 4.931927] ACPI: \_SB_.LNKA: Enabled at IRQ 11 [ 5.300976] ACPI: \_SB_.LNKC: Enabled at IRQ 11 [ 5.538396] ACPI: \_SB_.LNKD: Enabled at IRQ 10 [ 5.919157] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled [ 5.955850] 00:03: ttyS1 at I/O 0x2f8 (irq = 3, base_baud = 115200) is a 16550A [ 5.999200] 00:04: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A [ 6.007890] Non-volatile memory driver v1.3 [ 6.009310] Linux agpgart interface v0.103 [ 6.054177] virtio_blk virtio1: [vda] 134040 512-byte logical blocks (68.6 MB/65.4 MiB) [ 6.056459] vda: detected capacity change from 0 to 68628480 [ 6.071855] virtio_blk virtio2: [vdb] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) [ 6.077418] vdb: detected capacity change from 0 to 1073741824 [ 6.094779] virtio_blk virtio3: [vdc] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 6.100797] vdc: detected capacity change from 0 to 2621440000 [ 6.121521] virtio_blk virtio4: [vdd] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 6.124470] vdd: detected capacity change from 0 to 2621440000 [ 6.144849] virtio_blk virtio5: [vde] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 6.148858] vde: detected capacity change from 0 to 4294967296 [ 6.170744] virtio_blk virtio6: [vdf] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 6.175129] vdf: detected capacity change from 0 to 4294967296 [ 6.182947] libphy: Fixed MDIO Bus: probed [ 6.195086] usbcore: registered new interface driver usbserial_generic [ 6.197180] usbserial: USB Serial support registered for generic [ 6.199467] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12 [ 6.208199] serio: i8042 KBD port at 0x60,0x64 irq 1 [ 6.210098] serio: i8042 AUX port at 0x60,0x64 irq 12 [ 6.212586] mousedev: PS/2 mouse device common for all mice [ 6.216433] rtc_cmos 00:05: RTC can wake from S4 [ 6.222902] rtc_cmos 00:05: registered as rtc0 [ 6.226212] rtc_cmos 00:05: alarms up to one day, y3k, 242 bytes nvram, hpet irqs [ 6.228968] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input1 [ 6.233884] intel_pstate: CPU model not supported [ 6.237987] hid: raw HID events driver (C) Jiri Kosina [ 6.245444] usbcore: registered new interface driver usbhid [ 6.246857] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input4 [ 6.249789] usbhid: USB HID core driver [ 6.261433] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input3 [ 6.268236] drop_monitor: Initializing network drop monitor service [ 6.271644] Initializing XFRM netlink socket [ 6.273796] NET: Registered protocol family 10 [ 6.278259] Segment Routing with IPv6 [ 6.279363] NET: Registered protocol family 17 [ 6.281729] mpls_gso: MPLS GSO support [ 6.287227] RAS: Correctable Errors collector initialized. [ 6.289381] AVX version of gcm_enc/dec engaged. [ 6.290771] AES CTR mode by8 optimization enabled [ 6.451424] sched_clock: Marking stable (6451385143, 0)->(7635889275, -1184504132) [ 6.462647] registered taskstats version 1 [ 6.474922] Loading compiled-in X.509 certificates [ 6.476879] zswap: loaded using pool lzo/zbud [ 6.523618] Key type big_key registered [ 6.552914] Key type encrypted registered [ 6.555318] ima: No TPM chip found, activating TPM-bypass! [ 6.557055] ima: Allocated hash algorithm: sha1 [ 6.558419] ima: No architecture policies found [ 6.564449] evm: Initialising EVM extended attributes: [ 6.565742] evm: security.selinux [ 6.567095] evm: security.ima [ 6.575478] evm: security.capability [ 6.576310] evm: HMAC attrs: 0x1 [ 6.588378] rtc_cmos 00:05: setting system clock to 2025-12-29 18:54:59 UTC (1767034499) [ 6.593851] debug: unmapping init [mem 0xffffffffa7003000-0xffffffffa71fffff] [ 6.609512] debug: unmapping init [mem 0xffffffffa5d82000-0xffffffffa6058fff] [ 6.617069] Write protecting the kernel read-only data: 28672k [ 6.624332] debug: unmapping init [mem 0xffffffffa4403000-0xffffffffa45fffff] [ 6.630469] debug: unmapping init [mem 0xffffffffa4d14000-0xffffffffa4dfffff] [ 6.725980] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 6.739336] systemd[1]: Detected virtualization kvm. [ 6.741313] systemd[1]: Detected architecture x86-64. [ 6.742908] systemd[1]: Running in initial RAM disk. Welcome to Rocky Linux 8.10 (Green Obsidian) dracut-049-233.git20240115.el8 (Initramfs)! [ 6.788206] systemd[1]: No hostname configured. [ 6.794235] systemd[1]: Set hostname to . [ 6.795897] random: systemd: uninitialized urandom read (16 bytes read) [ 6.799758] systemd[1]: Initializing machine ID from random generator. [ 6.972807] random: ln: uninitialized urandom read (6 bytes read) [ 7.213690] random: systemd: uninitialized urandom read (16 bytes read) [ 7.218532] systemd[1]: Reached target Timers. [ OK ] Reached target Timers. [ 7.231944] systemd[1]: Reached target Swap. [ OK ] Reached target Swap. [ 7.240767] systemd[1]: Reached target Local File Systems. [ OK ] Reached target Local File Systems. [ OK ] Reached target Initrd Root Device. [ OK ] Listening on Journal Socket. Starting Apply Kernel Variables... [ OK ] Reached target Slices. [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Reached target Local Encrypted Volumes. [ OK ] Listening on udev Control Socket. [ OK ] Started Memstrack Anylazing Service. [ OK ] Reached target Paths. Starting Create list of required st…ce nodes for the current kernel... [ OK ] Listening on Journal Socket (/dev/log). Starting Journal Service... Starting Setup Virtual Console... [ OK ] Listening on udev Kernel Socket. [ OK ] Reached target Sockets. Starting Create Volatile Files and Directories... [ OK ] Started Apply Kernel Variables. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Started Setup Virtual Console. Starting dracut cmdline hook... Starting Create Static Device Nodes in /dev... [ OK ] Started Create Volatile Files and Directories. [ OK ] Started Create Static Device Nodes in /dev. [ OK ] Started Journal Service. [ OK ] Started dracut cmdline hook. Starting dracut pre-udev hook... [ 9.125931] device-mapper: uevent: version 1.0.3 [ 9.137619] device-mapper: ioctl: 4.46.0-ioctl (2022-02-22) initialised: dm-devel@redhat.com [ OK ] Started dracut pre-udev hook. Starting udev Kernel Device Manager... [ OK ] Started udev Kernel Device Manager. Starting dracut pre-trigger hook... [ OK ] Started dracut pre-trigger hook. Starting udev Coldplug all Devices... Mounting Kernel Configuration File System... [ OK ] Mounted Kernel Configuration File System. [ OK ] Started udev Coldplug all Devices. [ OK ] Reached target System Initialization. [ OK ] Reached target Basic System. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. Starting dracut initqueue hook... [ 10.815765] virtio_net virtio0 ens2: renamed from eth0 [ 10.899356] random: fast init done [ 11.482041] scsi host0: ata_piix [ 11.630219] scsi host1: ata_piix [ 11.631522] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc320 irq 14 [ 11.632977] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc328 irq 15 [ 17.108577] random: crng init done [ 17.109948] random: 7 urandom warning(s) missed due to ratelimiting [ 20.184279] dracut-initqueue[588]: RTNETLINK answers: File exists Starting nbd nbd0... [ OK ] Started nbd nbd0. [ OK ] Started dracut initqueue hook. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Mounting /sysroot... [ 21.469083] EXT4-fs (nbd0): mounted filesystem with ordered data mode. Opts: (null) [ OK ] Mounted /sysroot. [ OK ] Reached target Initrd Root File System. Starting Reload Configuration from the Real Root... [ OK ] Started Reload Configuration from the Real Root. [ OK ] Reached target Initrd File Systems. [ OK ] Reached target Initrd Default Target. Starting dracut pre-pivot and cleanup hook... [ OK ] Started dracut pre-pivot and cleanup hook. Starting Cleaning Up and Shutting Down Daemons... [ OK ] Stopped target Timers. Stopping Hardware RNG Entropy Gatherer Daemon... [ OK ] Stopped dracut pre-pivot and cleanup hook. [ OK ] Stopped target Remote File Systems. [ OK ] Stopped target Remote File Systems (Pre). [ OK ] Stopped target Initrd Default Target. [ OK ] Stopped target Initrd Root Device. [ OK ] Stopped dracut initqueue hook. [ OK ] Stopped Hardware RNG Entropy Gatherer Daemon. [ OK ] Stopped target Basic System. [ OK ] Stopped target Paths. [ OK ] Stopped target Sockets. [ OK ] Stopped target Slices. [ OK ] Stopped target System Initialization. [ OK ] Stopped target Local Encrypted Volumes. [ OK ] Stopped Dispatch Password Requests to Console Directory Watch. [ OK ] Stopped udev Coldplug all Devices. [ OK ] Stopped dracut pre-trigger hook. Stopping udev Kernel Device Manager... [ OK ] Stopped Apply Kernel Variables. [ OK ] Stopped target Swap. [ OK ] Stopped Create Volatile Files and Directories. [ OK ] Stopped target Local File Systems. [ OK ] Stopped udev Kernel Device Manager. [ OK ] Started Cleaning Up and Shutting Down Daemons. [ OK ] Stopped Create Static Device Nodes in /dev. [ OK ] Stopped Create list of required sta…vice nodes for the current kernel. [ OK ] Stopped dracut pre-udev hook. [ OK ] Stopped dracut cmdline hook. [ OK ] Closed udev Control Socket. [ OK ] Closed udev Kernel Socket. Starting Cleanup udevd DB... [ OK ] Started Cleanup udevd DB. [ OK ] Reached target Switch Root. Starting Switch Root... [ 24.338228] printk: systemd: 26 output lines suppressed due to ratelimiting [ 25.228135] SELinux: Disabled at runtime. [ 25.336920] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 25.360496] systemd[1]: Detected virtualization kvm. [ 25.363813] systemd[1]: Detected architecture x86-64. Welcome to Rocky Linux 8.10 (Green Obsidian)! [ 25.844562] hrtimer: interrupt took 6763171 ns [ 27.243742] systemd[1]: initrd-switch-root.service: Succeeded. [ 27.255857] systemd[1]: Stopped Switch Root. [ OK ] Stopped Switch Root. [ 27.269532] systemd[1]: systemd-journald.service: Service has no hold-off time (RestartSec=0), scheduling restart. [ 27.273366] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. [ 27.276599] systemd[1]: Stopped Journal Service. [ OK ] Stopped Journal Service. [ 27.315666] systemd[1]: Starting Journal Service... Starting Journal Service... [ 27.341770] systemd[1]: proc-sys-fs-binfmt_misc.automount: Refusing to start, unit to trigger not loaded. [FAILED] Failed to set up automount Arbitrar…rmats File System Automount Point. See 'systemctl status proc-sys-fs-binfmt_misc.automount' for details. [ OK ] Listening on initctl Compatibility Named Pipe. Starting Apply Kernel Variables... [ OK ] Listening on udev Control Socket. [ OK ] Stopped target Switch Root. [ OK ] Stopped target Initrd Root File System. [ OK ] Created slice system-getty.slice. Mounting Huge Pages File System... [ OK ] Started Forward Password Requests to Wall Directory Watch. Mounting POSIX Message Queue File System... Activating swap /dev/disk/by-label/SWAP... [ OK ] Created slice system-sshd\x2dkeygen.slice. [ OK ] Listening on RPCbind Server Activation Socket. [ OK ] Listening on Process Core Dump Socket. [ OK ] Started Disp[ 27.937647] Adding 1048572k swap on /dev/vdb. Priority:-2 extents:1 across:1048572k FS atch Password Requests to Console Directory Watch. [ OK ] Reached target Paths. [ OK ] Reached target Local Encrypted Volumes. [ OK ] Stopped target Initrd File Systems. Starting Remount Root and Kernel File Systems... [ OK ] Listening on udev Kernel Socket. Starting udev Coldplug all Devices... [ OK ] Created slice system-serial\x2dgetty.slice. Starting Create list of required st…ce nodes for the current kernel... [ OK ] Reached target RPC Port Mapper. [ OK ] Created slice User and Session Slice. [ OK ] Reached target Slices. Mounting Kernel Debug File System... [ OK ] Reached target rpc_pipefs.target. [ OK ] Started Journal Service. [ OK ] Started Apply Kernel Variables. [ OK ] Mounted Huge Pages File System. [ OK ] Mounted POSIX Message Queue File System. [ OK ] Activated swap /dev/disk/by-label/SWAP. [FAILED] Failed to start Remount Root and Kernel File Systems. See 'systemctl status systemd-remount-fs.service' for details. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Mounted Kernel Debug File System. Starting Configure read-only root support... Starting Create Static Device Nodes in /dev... [ OK ] Reached target Swap. Starting Flush Journal to Persistent Storage... [ OK ] Started Flush Journal to Persistent Storage. [ OK ] Started Create Static Device Nodes in /dev. Starting udev Kernel Device Manager... [ OK ] Reached target Local File Systems (Pre). Mounting /home/green/git/lustre-release... Mounting /mnt... [ OK ] Started udev Coldplug all Devices. [ OK ] Mounted /mnt. [ 29.082421] squashfs: version 4.0 (2009/01/31) Phillip Lougher [ OK ] Mounted /home/green/git/lustre-release. [ OK ] Started udev Kernel Device Manager. [ 30.291203] input: PC Speaker as /devices/platform/pcspkr/input/input5 [ 30.341234] piix4_smbus 0000:00:01.3: SMBus Host Controller at 0x700, revision 0 [ 32.419447] RAPL PMU: API unit is 2^-32 Joules, 0 fixed counters, 10737418240 ms ovfl timer [ 32.670425] EDAC sbridge: Ver: 1.1.2 [* ] A start job is running for Configur…-only root support (8s / no limit)[ 35.332720] Key type dns_resolver registered [** ] A start job is running for Configur…-only root support (8s / no limit) [*** ] A start job is running for Configur…-only root support (9s / no limit)[ 36.446130] NFS: Registering the id_resolver key type [ 36.457650] Key type id_resolver registered [ 36.459014] Key type id_legacy registered [ *** ] A start job is running for Configur…-only root support (9s / no limit) [ *** ] A start job is running for Configur…only root support (10s / no limit) [ ***] A start job is running for Configur…only root support (10s / no limit) [ **] A start job is running for Configur…only root support (10s / no limit) [ OK ] Started Configure read-only root support. [ OK ] Reached target Local File Systems. Starting Create Volatile Files and Directories... Starting Mark the need to relabel after reboot... Starting Rebuild Dynamic Linker Cache... Starting Load/Save Random Seed... [ OK ] Started Mark the need to relabel after reboot. [ OK ] Started Load/Save Random Seed. [ OK ] Started Create Volatile Files and Directories. Starting RPC Bind... Starting Update UTMP about System Boot/Shutdown... [ OK ] Started Update UTMP about System Boot/Shutdown. [ OK ] Started RPC Bind. [ OK ] Started Rebuild Dynamic Linker Cache. Starting Update is Completed... [ OK ] Started Update is Completed. [ OK ] Reached target System Initialization. [ OK ] Listening on D-Bus System Message Bus Socket. [ OK ] Reached target Sockets. [ OK ] Reached target Basic System. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. Starting Login Service... [ OK ] Started D-Bus System Message Bus. Starting Network Manager... [ OK ] Started Daily Cleanup of Temporary Directories. [ OK ] Started daily update of the root trust anchor for DNSSEC. Starting Restore /run/initramfs on shutdown... [ OK ] Started dnf makecache --timer. [ OK ] Reached target Timers. [ OK ] Started irqbalance daemon. [ OK ] Reached target sshd-keygen.target. [ OK ] Started Restore /run/initramfs on shutdown. [ OK ] Started Network Manager. [ OK ] Reached target Network. Starting Dynamic System Tuning Daemon... Starting OpenSSH server daemon... Starting GSSAPI Proxy Daemon... Starting Network Manager Wait Online... [ OK ] Started OpenSSH server daemon. [ OK ] Started GSSAPI Proxy Daemon. Starting Hostname Service... [ OK ] Reached target NFS client services. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Starting Permit User Sessions... [ OK ] Started Permit User Sessions. [ OK ] Started Serial Getty on ttyS0. [ OK ] Started Getty on tty1. [ OK ] Started Command Scheduler. [ OK ] Started Serial Getty on ttyS1. [ OK ] Reached target Login Prompts. [ OK ] Started Login Service. [ OK ] Started Hostname Service. Starting Network Manager Script Dispatcher Service... [ OK ] Started Network Manager Script Dispatcher Service. [ OK ] Started Network Manager Wait Online. [ OK ] Reached target Network is Online. Starting Notify NFS peers of a restart... Starting Crash recovery kernel arming... Starting System Logging Service... [ OK ] Started Notify NFS peers of a restart. [ OK ] Started System Logging Service. Rocky Linux 8.10 (Green Obsidian) Kernel 4.18.0rh8.10-debug on an x86_64 oleg429-server login: [ 65.875770] libcfs: loading out-of-tree module taints kernel. [ 65.895024] Key type ._llcrypt registered [ 65.896450] Key type .llcrypt registered [ 65.942347] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_hostid [ 73.355837] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing load_modules_local [ 73.957016] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 73.963878] alg: No test for adler32 (adler32-zlib) [ 74.976260] Lustre: Lustre: Build Version: 2.17.0_RC3_1_ga0d6154 [ 75.252648] LNet: Added LNI 192.168.204.129@tcp [8/256/0/180] [ 76.856155] Key type lgssc registered [ 77.394610] Lustre: Echo OBD driver; http://www.lustre.org/ [ 84.541531] ZFS: Loaded module v2.3.2-1, ZFS pool version 5000, ZFS filesystem version 5 [ 84.963216] LDISKFS-fs (vdc): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 88.198113] LDISKFS-fs (vdd): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 91.033545] LDISKFS-fs (vde): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 94.091683] LDISKFS-fs (vdf): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 100.393168] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing load_modules_local [ 105.431858] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 105.455604] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 105.465260] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 106.568367] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 106.580234] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 106.617083] Lustre: lustre-MDT0000: new disk, initializing [ 106.644421] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 106.652156] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 108.072088] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 113.624854] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 113.651632] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 113.681323] Lustre: 6481:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 113.695531] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 113.698173] Lustre: Skipped 1 previous similar message [ 113.740320] Lustre: lustre-MDT0001: new disk, initializing [ 113.771717] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 113.784680] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 113.789190] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 115.588375] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 118.539112] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 125.328923] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 125.404067] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 125.568245] Lustre: lustre-OST0000: new disk, initializing [ 125.571547] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 125.651391] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 130.023174] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 130.065916] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 130.070228] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 130.125544] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 139.945914] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 140.002700] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 140.103732] Lustre: lustre-OST0001: new disk, initializing [ 140.106770] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 140.181136] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 145.065798] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 146.447204] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 146.457199] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 146.492056] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 152.552678] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 158.698829] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 166.106222] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing check_logdir /tmp/testlogs/ [ 168.439968] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing yml_node [ 170.774542] Lustre: DEBUG MARKER: Client: 2.17.0.RC3 [ 172.219311] Lustre: DEBUG MARKER: MDS: 2.17.0.RC3 [ 173.482982] Lustre: DEBUG MARKER: OSS: 2.17.0.RC3 [ 174.217937] Lustre: DEBUG MARKER: -----============= acceptance-small: conf-sanity ============----- Mon Dec 29 13:57:46 EST 2025 [ 181.855588] Lustre: DEBUG MARKER: excepting tests: 102 106 115 32newtarball [ 182.559228] Lustre: DEBUG MARKER: skipping tests SLOW=no: 45 69 106 111 114 [ 196.064552] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 196.067793] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 196.073622] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 197.600990] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 197.604328] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 197.606570] Lustre: Skipped 1 previous similar message [ 197.612300] Lustre: Skipped 3 previous similar messages [ 200.848524] LustreError: 12182:0:(obd_class.h:479:obd_check_dev()) Device 28 not setup [ 200.891982] Lustre: server umount lustre-MDT0000 complete [ 202.595700] LustreError: 7414:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1767034695 with bad export cookie 10880436190458302138 [ 202.597353] LustreError: MGC192.168.204.129@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 202.600565] LustreError: 7414:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 202.720722] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 202.721062] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 202.726604] Lustre: Skipped 2 previous similar messages [ 207.847277] LustreError: 10422:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 207.850781] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 207.859712] LustreError: 10422:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 4 previous similar messages [ 207.866386] Lustre: Skipped 2 previous similar messages [ 208.719278] LustreError: 12384:0:(obd_class.h:479:obd_check_dev()) Device 27 not setup [ 208.721378] LustreError: 12384:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 208.809624] Lustre: server umount lustre-MDT0001 complete [ 210.621852] LustreError: 12586:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 210.623751] LustreError: 12586:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 210.651037] Lustre: server umount lustre-OST0000 complete [ 212.425463] Lustre: server umount lustre-OST0001 complete [ 215.155755] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_hostid [ 219.654336] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing load_modules_local [ 224.531915] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 227.758801] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 230.793610] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 234.122460] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 237.735711] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 237.779248] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 237.913894] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 237.941958] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 237.998940] Lustre: lustre-MDT0000: new disk, initializing [ 238.040675] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 238.050193] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 240.057970] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 245.854154] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 245.900317] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 245.956586] Lustre: 14965:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 246.003090] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 246.005804] Lustre: Skipped 1 previous similar message [ 246.055258] Lustre: lustre-MDT0001: new disk, initializing [ 246.109859] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 246.134028] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 246.139627] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 248.224680] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 251.950977] Lustre: DEBUG MARKER: oleg429-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 253.659707] Lustre: DEBUG MARKER: oleg429-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 256.805882] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 256.850788] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 258.314279] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 258.389531] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 259.773229] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 263.633378] Lustre: DEBUG MARKER: oleg429-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 265.429927] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 265.532585] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 267.244703] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 267.337188] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 268.769335] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 268.770762] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 268.772576] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 268.778246] Lustre: Skipped 1 previous similar message [ 268.783727] Lustre: Skipped 1 previous similar message [ 274.226257] LustreError: 17900:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 274.228488] LustreError: 17900:0:(obd_class.h:479:obd_check_dev()) Skipped 9 previous similar messages [ 274.285996] Lustre: server umount lustre-OST0000 complete [ 276.961102] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 276.964146] LustreError: Skipped 1 previous similar message [ 276.965682] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 276.972528] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 276.975324] Lustre: Skipped 2 previous similar messages [ 290.272129] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 290.329799] LustreError: 18102:0:(obd_class.h:479:obd_check_dev()) Device 22 not setup [ 290.331731] LustreError: 18102:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 290.380589] Lustre: server umount lustre-MDT0000 complete [ 292.217307] LustreError: 15755:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1767034785 with bad export cookie 10880436190458303650 [ 292.220473] LustreError: MGC192.168.204.129@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 292.221454] LustreError: 15755:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 292.385746] Lustre: server umount lustre-MDT0001 complete [ 294.926526] Lustre: DEBUG MARKER: == conf-sanity test 76a: set permanent params with lctl across mounts ========================================================== 13:59:46 (1767034786) [ 298.168286] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 298.394543] LustreError: 18830:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 298.438452] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 298.441586] Lustre: Skipped 1 previous similar message [ 300.214704] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 303.586547] LustreError: 18829:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 304.166617] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 306.396222] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 308.911955] Lustre: DEBUG MARKER: oleg429-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 310.704089] Lustre: DEBUG MARKER: oleg429-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 314.313750] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 317.338188] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 319.930359] Lustre: DEBUG MARKER: oleg429-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 327.523986] Lustre: Modifying parameter general.osc.*.max_dirty_mb=452 in log params [ 334.307757] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 334.309765] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 334.318200] Lustre: Skipped 3 previous similar messages [ 334.322976] Lustre: Skipped 7 previous similar messages [ 337.357847] LustreError: 21131:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup [ 337.360295] LustreError: 21131:0:(obd_class.h:479:obd_check_dev()) Skipped 13 previous similar messages [ 337.402354] Lustre: server umount lustre-MDT0000 complete [ 339.377845] LustreError: 20448:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1767034832 with bad export cookie 10880436190458304539 [ 339.380565] LustreError: MGC192.168.204.129@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 339.383246] LustreError: 20448:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 339.426220] LustreError: 19942:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 339.576642] Lustre: server umount lustre-MDT0001 complete [ 347.804448] Lustre: server umount lustre-OST0000 complete [ 359.280120] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing load_modules_local [ 365.402847] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 365.643666] LustreError: 22861:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 365.697510] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 365.700809] Lustre: Skipped 2 previous similar messages [ 368.033098] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 373.339158] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 375.754088] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 377.484799] Lustre: Setting parameter general.debug_raw_pointers=Y in log params [ 381.689027] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 385.333331] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 390.066643] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 390.135040] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 390.227695] Lustre: lustre-OST0001: new disk, initializing [ 390.229827] Lustre: Skipped 1 previous similar message [ 390.232549] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 390.235330] Lustre: Skipped 1 previous similar message [ 391.830187] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 391.833533] Lustre: Skipped 1 previous similar message [ 391.835888] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 391.891846] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 393.358583] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 399.886724] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 402.152634] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 414.691553] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 414.695045] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 414.699714] Lustre: Skipped 1 previous similar message [ 414.703625] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 414.706634] Lustre: Skipped 2 previous similar messages [ 416.080990] LustreError: 26352:0:(obd_class.h:479:obd_check_dev()) Device 28 not setup [ 416.084364] LustreError: 26352:0:(obd_class.h:479:obd_check_dev()) Skipped 17 previous similar messages [ 416.136895] Lustre: server umount lustre-MDT0000 complete [ 417.765838] LustreError: 22861:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 417.780096] LustreError: 22861:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 4 previous similar messages [ 418.256822] LustreError: 25893:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1767034911 with bad export cookie 10880436190458305386 [ 418.261837] LustreError: MGC192.168.204.129@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 418.270349] LustreError: 25893:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 420.461517] Lustre: server umount lustre-MDT0001 complete [ 432.148584] Lustre: server umount lustre-OST0000 complete [ 444.830584] Lustre: server umount lustre-OST0001 complete [ 454.227062] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing load_modules_local [ 459.730265] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 459.961692] LustreError: 28129:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 460.002060] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 460.005435] Lustre: Skipped 3 previous similar messages [ 462.128431] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 466.892871] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 469.314969] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 470.953925] Lustre: 29246:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 474.873342] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 477.773476] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 481.704934] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 484.791945] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 489.178964] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 491.048712] Lustre: 31100:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 500.196070] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 500.197118] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 500.202266] Lustre: Skipped 5 previous similar messages [ 500.210076] Lustre: Skipped 3 previous similar messages [ 504.977554] LustreError: 31303:0:(obd_class.h:479:obd_check_dev()) Device 10 not setup [ 504.979782] LustreError: 31303:0:(obd_class.h:479:obd_check_dev()) Skipped 25 previous similar messages [ 505.046283] Lustre: server umount lustre-MDT0000 complete [ 505.314147] LustreError: 28125:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 505.326022] LustreError: 28125:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 5 previous similar messages [ 506.949978] LustreError: 29628:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1767034999 with bad export cookie 10880436190458306681 [ 506.950485] LustreError: MGC192.168.204.129@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 506.954761] LustreError: 29628:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 507.100769] Lustre: server umount lustre-MDT0001 complete [ 519.705730] Lustre: server umount lustre-OST0000 complete [ 531.962663] Lustre: server umount lustre-OST0001 complete [ 536.626411] Lustre: DEBUG MARKER: == conf-sanity test 76b: verify params log setup correctly ========================================================== 14:03:48 (1767035028) [ 550.276090] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing load_modules_local [ 555.411041] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 557.834967] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 562.251968] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 564.505678] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 566.042645] Lustre: 34998:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 569.454037] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 572.481105] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 572.711814] LustreError: 35354:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 572.724205] LustreError: 35354:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 576.606105] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 579.761349] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 583.800676] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 585.328855] Lustre: 36851:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 593.377640] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 593.381903] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 593.391152] Lustre: Skipped 1 previous similar message [ 598.089221] LustreError: 37003:0:(obd_class.h:479:obd_check_dev()) Device 10 not setup [ 598.091987] LustreError: 37003:0:(obd_class.h:479:obd_check_dev()) Skipped 25 previous similar messages [ 598.121673] Lustre: server umount lustre-MDT0000 complete [ 599.630901] LustreError: 33862:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1767035092 with bad export cookie 10880436190458307787 [ 599.632472] LustreError: MGC192.168.204.129@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 599.635259] LustreError: 33862:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 599.763414] Lustre: server umount lustre-MDT0001 complete [ 611.793556] Lustre: server umount lustre-OST0000 complete [ 623.850972] Lustre: server umount lustre-OST0001 complete [ 627.491652] Lustre: DEBUG MARKER: == conf-sanity test 76c: verify changelog_mask is applied with lctl set_param -P ========================================================== 14:05:19 (1767035119) [ 634.326597] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing load_modules_local [ 639.208823] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 639.451647] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 639.454743] Lustre: Skipped 7 previous similar messages [ 641.607674] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 645.749104] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 647.601484] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 648.932706] Lustre: 40091:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 652.071992] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 654.943652] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 659.117671] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 661.850564] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 670.572509] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 677.258044] Lustre: Setting parameter general.mdd.*.changelog_mask=-CLOSE in log params [ 677.260678] Lustre: Skipped 1 previous similar message [ 680.426830] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 680.429537] Lustre: Skipped 8 previous similar messages [ 684.544641] Lustre: server umount lustre-MDT0000 complete [ 686.427751] LustreError: 38955:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1767035179 with bad export cookie 10880436190458308893 [ 686.429388] LustreError: MGC192.168.204.129@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 686.433318] LustreError: 38955:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 692.635956] Lustre: server umount lustre-MDT0001 complete [ 696.464359] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 696.801740] LustreError: 3626:0:(client.c:1390:ptlrpc_import_delay_req()) @@@ invalidate in flight req@ffff9d02066e9180 x1852870039503744/t0(0) o250->MGC192.168.204.129@tcp@0@lo:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl Rpc:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 698.705583] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 701.026699] LustreError: 40459:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 701.036103] LustreError: 40459:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 14 previous similar messages [ 702.577961] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 704.572264] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 707.184262] Lustre: DEBUG MARKER: oleg429-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 715.063434] Lustre: DEBUG MARKER: mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 7 sec [ 716.267275] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 716.458323] Lustre: DEBUG MARKER: oleg429-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 717.075940] Lustre: DEBUG MARKER: mdc.lustre-MDT0001-mdc-*.mds_server_uuid in FULL state after 0 sec [ 721.381241] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 721.387361] Lustre: Skipped 10 previous similar messages [ 724.899377] Lustre: server umount lustre-MDT0000 complete [ 726.388994] LustreError: 41953:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1767035219 with bad export cookie 10880436190458310034 [ 726.390773] LustreError: MGC192.168.204.129@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 726.393411] LustreError: 41953:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 1 previous similar message [ 726.473854] LustreError: 44545:0:(obd_class.h:479:obd_check_dev()) Device 18 not setup [ 726.476384] LustreError: 44545:0:(obd_class.h:479:obd_check_dev()) Skipped 49 previous similar messages [ 726.535750] Lustre: server umount lustre-MDT0001 complete [ 737.165820] Lustre: server umount lustre-OST0000 complete [ 749.113848] Lustre: server umount lustre-OST0001 complete [ 752.762984] Lustre: DEBUG MARKER: == conf-sanity test 76d: verify llite.*.xattr_cache can be set by 'lctl set_param -P' correctly ========================================================== 14:07:24 (1767035244) [ 759.017419] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing load_modules_local [ 763.337226] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 765.421625] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 769.512285] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 771.508692] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 772.954628] Lustre: 47434:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 772.959418] Lustre: 47434:0:(mgs_llog.c:1348:mgs_modify_param()) Skipped 1 previous similar message [ 776.356249] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 779.769943] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 784.171619] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 786.784862] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 795.743637] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 802.675145] Lustre: Setting parameter general.llite.*.xattr_cache=0 in log params [ 813.637690] Lustre: server umount lustre-MDT0000 complete [ 815.375774] LustreError: 49292:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1767035308 with bad export cookie 10880436190458310832 [ 815.380555] LustreError: MGC192.168.204.129@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 815.384483] LustreError: 49292:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 815.517948] Lustre: server umount lustre-MDT0001 complete [ 827.412754] Lustre: server umount lustre-OST0000 complete [ 838.242391] Lustre: server umount lustre-OST0001 complete [ 841.973313] Lustre: DEBUG MARKER: == conf-sanity test 77: comma-separated MGS NIDs and failover node NIDs ========================================================== 14:08:54 (1767035334) [ 842.885547] Lustre: DEBUG MARKER: SKIP: conf-sanity test_77 mixed loopback and real device not working [ 843.601045] Lustre: DEBUG MARKER: == conf-sanity test 78: run resize2fs on MDT and OST filesystems ========================================================== 14:08:55 (1767035335) [ 847.743799] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_hostid [ 850.793400] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing load_modules_local [ 854.500250] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 856.751046] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 859.219676] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 859.246546] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 859.362826] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 859.402808] Lustre: lustre-MDT0000: new disk, initializing [ 859.438650] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 860.891284] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 864.160138] Lustre: DEBUG MARKER: oleg429-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 866.798767] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 866.827102] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 868.163359] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 868.174859] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 869.103039] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 872.302748] Lustre: DEBUG MARKER: oleg429-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 872.996625] Lustre: DEBUG MARKER: create test files [ 888.801704] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 890.450423] Lustre: server umount lustre-OST0000 complete [ 891.980910] Lustre: server umount lustre-MDT0000 complete [ 896.028555] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing unload_modules_local [ 897.178852] Key type lgssc unregistered [ 897.318779] LNet: 54837:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 897.322138] LNetError: 54837:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 897.331510] LNet: Removed LNI 192.168.204.129@tcp [ 897.714267] Key type .llcrypt unregistered [ 897.715642] Key type ._llcrypt unregistered [ 909.530747] Key type ._llcrypt registered [ 909.532250] Key type .llcrypt registered [ 909.582318] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing load_modules_local [ 909.995499] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 910.030560] alg: No test for adler32 (adler32-zlib) [ 910.945843] Lustre: Lustre: Build Version: 2.17.0_RC3_1_ga0d6154 [ 911.051140] LNet: Added LNI 192.168.204.129@tcp [8/256/0/180] [ 912.640210] Key type lgssc registered [ 913.081921] Lustre: Echo OBD driver; http://www.lustre.org/ [ 916.219031] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 916.230989] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 917.446799] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 918.999900] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 921.031302] Lustre: DEBUG MARKER: oleg429-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 923.594742] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 923.705472] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 925.649276] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 927.650060] Lustre: DEBUG MARKER: oleg429-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 927.781062] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:102 to 0x240000400:129) [ 927.812794] Lustre: 57421:0:(osd_internal.h:1459:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 515 < left 544, rollback = 9 [ 927.816264] Lustre: 57421:0:(osd_handler.c:2096:osd_trans_dump_creds()) create: 32/128/0, destroy: 0/0/0 [ 927.819840] Lustre: 57421:0:(osd_handler.c:2103:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 927.823061] Lustre: 57421:0:(osd_handler.c:2113:osd_trans_dump_creds()) write: 2/13/1, punch: 0/0/0, quota 1/3/0 [ 927.825919] Lustre: 57421:0:(osd_handler.c:2120:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 927.829219] Lustre: 57421:0:(osd_handler.c:2127:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 930.814249] Lustre: DEBUG MARKER: check files after expanding the MDT and OST filesystems [ 932.874063] Lustre: DEBUG MARKER: create more files after expanding the MDT and OST filesystems [ 937.955056] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 937.955513] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 937.963472] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 941.103086] LustreError: 58164:0:(obd_class.h:479:obd_check_dev()) Device 13 not setup [ 941.146172] Lustre: server umount lustre-OST0000 complete [ 942.672838] LustreError: 58366:0:(obd_class.h:479:obd_check_dev()) Device 8 not setup [ 942.675368] LustreError: 58366:0:(obd_class.h:479:obd_check_dev()) Skipped 1 previous similar message [ 942.739788] Lustre: server umount lustre-MDT0000 complete [ 946.862738] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing unload_modules_local [ 948.012239] Key type lgssc unregistered [ 948.155272] LNet: 58898:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 948.158920] LNetError: 58898:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 948.166475] LNet: Removed LNI 192.168.204.129@tcp [ 948.523799] Key type .llcrypt unregistered [ 948.525416] Key type ._llcrypt unregistered [ 960.107981] Key type ._llcrypt registered [ 960.109438] Key type .llcrypt registered [ 960.151443] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing load_modules_local [ 960.523510] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 960.576212] alg: No test for adler32 (adler32-zlib) [ 961.439717] Lustre: Lustre: Build Version: 2.17.0_RC3_1_ga0d6154 [ 961.529430] LNet: Added LNI 192.168.204.129@tcp [8/256/0/180] [ 963.112120] Key type lgssc registered [ 963.481108] Lustre: Echo OBD driver; http://www.lustre.org/ [ 966.130044] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 966.138361] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 967.307805] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 968.630214] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 970.484531] Lustre: DEBUG MARKER: oleg429-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 972.870856] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 972.986648] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 974.914724] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 976.761451] Lustre: DEBUG MARKER: oleg429-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 978.085029] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:140 to 0x240000400:161) [ 978.103917] Lustre: 61580:0:(osd_internal.h:1459:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 515 < left 544, rollback = 9 [ 978.106856] Lustre: 61580:0:(osd_handler.c:2096:osd_trans_dump_creds()) create: 32/128/0, destroy: 0/0/0 [ 978.109257] Lustre: 61580:0:(osd_handler.c:2103:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 978.112155] Lustre: 61580:0:(osd_handler.c:2113:osd_trans_dump_creds()) write: 2/13/1, punch: 0/0/0, quota 1/3/0 [ 978.114770] Lustre: 61580:0:(osd_handler.c:2120:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 978.117654] Lustre: 61580:0:(osd_handler.c:2127:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 985.270433] Lustre: DEBUG MARKER: check files after shrinking the MDT and OST filesystems [ 988.641603] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 988.641833] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 988.650097] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 989.936086] LustreError: 62274:0:(obd_class.h:479:obd_check_dev()) Device 13 not setup [ 989.974918] Lustre: server umount lustre-OST0000 complete [ 993.405944] LustreError: 62475:0:(obd_class.h:479:obd_check_dev()) Device 8 not setup [ 993.407337] LustreError: 62475:0:(obd_class.h:479:obd_check_dev()) Skipped 1 previous similar message [ 993.463552] Lustre: server umount lustre-MDT0000 complete [ 997.167567] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing unload_modules_local [ 998.228919] Key type lgssc unregistered [ 998.362407] LNet: 63007:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 998.365656] LNetError: 63007:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 998.378306] LNet: Removed LNI 192.168.204.129@tcp [ 998.693110] Key type .llcrypt unregistered [ 998.694201] Key type ._llcrypt unregistered [ 1005.160842] Key type ._llcrypt registered [ 1005.162423] Key type .llcrypt registered [ 1005.215063] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_hostid [ 1011.901966] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing load_modules_local [ 1012.281256] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1012.337349] alg: No test for adler32 (adler32-zlib) [ 1013.209373] Lustre: Lustre: Build Version: 2.17.0_RC3_1_ga0d6154 [ 1013.306387] LNet: Added LNI 192.168.204.129@tcp [8/256/0/180] [ 1014.904174] Key type lgssc registered [ 1015.313448] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1018.626789] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1020.931140] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1023.102993] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1025.478650] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1028.015192] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1028.039246] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1028.043814] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1029.136368] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 1029.149529] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1029.191130] Lustre: lustre-MDT0000: new disk, initializing [ 1029.222327] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1029.230056] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1030.697978] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 1035.219414] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1035.241826] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1035.281110] Lustre: 66108:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 1035.317495] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 1035.320105] Lustre: Skipped 1 previous similar message [ 1035.356605] Lustre: lustre-MDT0001: new disk, initializing [ 1035.390807] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 1035.406124] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 1035.409844] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 1036.792676] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 1040.189176] Lustre: DEBUG MARKER: oleg429-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1041.545350] Lustre: DEBUG MARKER: oleg429-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 1043.972192] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1043.994072] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1044.090784] Lustre: lustre-OST0000: new disk, initializing [ 1044.093235] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1044.124641] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1046.190248] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 1049.327606] Lustre: DEBUG MARKER: oleg429-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1049.595657] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 1049.599208] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 1049.633793] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 1050.718793] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 1050.796098] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 1052.195644] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 1052.278424] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 1054.691125] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1054.693557] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1054.693874] LustreError: Skipped 1 previous similar message [ 1054.699006] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1054.703230] Lustre: Skipped 1 previous similar message [ 1059.056078] LustreError: 69047:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 1059.095417] Lustre: server umount lustre-OST0000 complete [ 1061.344913] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1061.347222] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1061.350542] Lustre: Skipped 1 previous similar message [ 1061.352500] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1064.929692] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1064.930706] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1066.504946] LustreError: 69248:0:(obd_class.h:479:obd_check_dev()) Device 22 not setup [ 1066.507566] LustreError: 69248:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 1066.544502] Lustre: server umount lustre-MDT0000 complete [ 1067.937763] LustreError: 66100:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1767035560 with bad export cookie 11634147280590983420 [ 1067.940479] LustreError: MGC192.168.204.129@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1067.942642] LustreError: 66100:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1068.011312] LustreError: 69449:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 1068.013671] LustreError: 69449:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 1068.067556] Lustre: server umount lustre-MDT0001 complete [ 1071.270571] Lustre: DEBUG MARKER: == conf-sanity test 79: format MDT/OST without mgs option (should return errors) ========================================================== 14:12:43 (1767035563) [ 1075.226470] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing load_modules_local [ 1079.926354] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1088.978784] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_hostid [ 1092.529194] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing load_modules_local [ 1096.289654] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1098.648484] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1100.908188] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1103.215200] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1105.621615] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1105.648187] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1105.733665] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 1105.746452] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1105.785560] Lustre: lustre-MDT0000: new disk, initializing [ 1105.813605] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1105.822160] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1107.093810] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 1111.422926] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1111.445691] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1111.480760] Lustre: 73677:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 1111.504827] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 1111.506466] Lustre: Skipped 1 previous similar message [ 1111.542497] Lustre: lustre-MDT0001: new disk, initializing [ 1111.580785] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 1111.593511] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 1111.597162] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 1112.979508] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 1116.198272] Lustre: DEBUG MARKER: oleg429-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1117.554270] Lustre: DEBUG MARKER: oleg429-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 1119.995837] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1120.022062] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1120.118430] Lustre: lustre-OST0000: new disk, initializing [ 1120.120770] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1120.150677] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1120.397514] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 1120.400434] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 1120.433556] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 1122.104835] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 1125.409015] Lustre: DEBUG MARKER: oleg429-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1126.916124] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 1127.000776] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 1128.433143] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 1128.517473] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 1130.978146] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1130.978976] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1130.983731] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1130.986583] LustreError: Skipped 1 previous similar message [ 1130.989449] Lustre: Skipped 1 previous similar message [ 1135.342177] LustreError: 76614:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 1135.344963] LustreError: 76614:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 1135.390459] Lustre: server umount lustre-OST0000 complete [ 1137.122149] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1137.125758] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1137.131429] Lustre: Skipped 1 previous similar message [ 1137.134071] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1137.137150] Lustre: Skipped 1 previous similar message [ 1142.910296] LustreError: 76817:0:(obd_class.h:479:obd_check_dev()) Device 22 not setup [ 1142.912125] LustreError: 76817:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 1142.935711] Lustre: server umount lustre-MDT0000 complete [ 1144.276402] LustreError: 73669:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1767035637 with bad export cookie 11634147280590984302 [ 1144.279733] LustreError: MGC192.168.204.129@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1144.281161] LustreError: 73669:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 1144.411678] Lustre: server umount lustre-MDT0001 complete [ 1147.456903] Lustre: DEBUG MARKER: == conf-sanity test 80: mgc import reconnect race ======== 14:13:59 (1767035639) [ 1149.852203] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1150.029507] LustreError: 77689:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1150.036847] LustreError: 77689:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 1150.056214] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1151.496847] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 1154.660826] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1156.267755] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 1158.225975] Lustre: DEBUG MARKER: oleg429-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1159.580371] Lustre: DEBUG MARKER: oleg429-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 1162.073930] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1164.315915] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 1166.256912] Lustre: DEBUG MARKER: oleg429-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1167.580918] Lustre: 80028:0:(genops.c:1791:obd_export_evict_by_uuid()) MGS: evicting 5d1d5044-8920-4d4f-842c-ffedb152e2dc at adminstrative request [ 1169.378494] LustreError: MGC192.168.204.129@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1169.383351] Lustre: *** cfs_fail_loc=906, val=2147483648*** [ 1179.620445] Lustre: MGS: Client 5d1d5044-8920-4d4f-842c-ffedb152e2dc (at 0@lo) reconnecting [ 1179.622861] Lustre: Evicted from MGS (at 0@lo) after server handle changed from 0xa174cb4fb433d7d2 to 0xa174cb4fb433da5d [ 1179.628705] Lustre: MGC192.168.204.129@tcp: Connection restored to 0@lo (at 0@lo) [ 1200.212384] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1200.231232] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1200.290691] Lustre: lustre-OST0001: new disk, initializing [ 1200.293374] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 1200.325113] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 1200.327052] Lustre: Skipped 2 previous similar messages [ 1202.313713] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 1205.503632] Lustre: DEBUG MARKER: oleg429-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid [ 1206.487078] LustreError: 81151:0:(obd_class.h:479:obd_check_dev()) Device 25 not setup [ 1206.488783] LustreError: 81151:0:(obd_class.h:479:obd_check_dev()) Skipped 13 previous similar messages [ 1206.504155] Lustre: server umount lustre-OST0001 complete [ 1209.841686] LustreError: 79286:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1209.842504] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1209.844438] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1209.844446] Lustre: Skipped 2 previous similar messages [ 1209.856183] Lustre: Skipped 2 previous similar messages [ 1214.947374] LustreError: 79286:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1214.953167] LustreError: 79286:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 1220.067788] LustreError: 80051:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1220.073166] LustreError: 80051:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 1222.112145] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 1222.160810] Lustre: server umount lustre-OST0000 complete [ 1229.632256] LustreError: 81561:0:(obd_class.h:479:obd_check_dev()) Device 24 not setup [ 1229.634140] LustreError: 81561:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 1229.661424] Lustre: server umount lustre-MDT0000 complete [ 1230.305513] LustreError: 80550:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1230.310710] LustreError: 80550:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 1230.952437] LustreError: 77668:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1767035723 with bad export cookie 11634147280590985821 [ 1230.956133] LustreError: MGC192.168.204.129@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1230.957524] LustreError: 77668:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1231.086221] Lustre: server umount lustre-MDT0001 complete [ 1233.956456] Lustre: DEBUG MARKER: == conf-sanity test 81: sparse OST indexing ============== 14:15:26 (1767035726) [ 1234.523779] Lustre: DEBUG MARKER: SKIP: conf-sanity test_81 needs >= 3 OSTs [ 1235.108373] Lustre: DEBUG MARKER: == conf-sanity test 82a: specify OSTs for file (succeed) or directory (succeed) ========================================================== 14:15:27 (1767035727) [ 1235.611714] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82a needs >= 3 OSTs [ 1236.193319] Lustre: DEBUG MARKER: == conf-sanity test 82b: specify OSTs for file with --pool and --ost-list options ========================================================== 14:15:28 (1767035728) [ 1236.724901] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82b needs >= 4 OSTs [ 1237.278482] Lustre: DEBUG MARKER: == conf-sanity test 82c: specify sparse OSTs for setstripe (should not crash) ========================================================== 14:15:29 (1767035729) [ 1237.791576] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82c needs >= 4 OSTs [ 1238.364050] Lustre: DEBUG MARKER: == conf-sanity test 83: ENOSPACE on OST doesn't cause message VFS: Busy inodes after unmount ... ========================================================== 14:15:30 (1767035730) [ 1238.885492] Lustre: DEBUG MARKER: mount the OST /dev/mapper/ost1_flakey as a ldiskfs filesystem [ 1240.605846] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1240.953809] loop: module loaded [ 1240.958520] loop0: detected capacity change from 0 to 4294967296 [ 1240.973924] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 1241.483322] Lustre: DEBUG MARKER: run llverfs in partial mode on the OST ldiskfs /mnt/lustre-ost1 [ 1242.498023] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing run_llverfs /mnt/lustre-ost1 -vpl no [ 1243.846088] Lustre: DEBUG MARKER: unmount the OST /dev/mapper/ost1_flakey [ 1246.688205] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1248.027939] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1249.315057] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1250.568920] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1252.648139] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1252.668455] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1252.679966] LustreError: 84424:0:(obd_config.c:783:class_setup()) setup lustre-OST0000-osd failed (-28) [ 1252.682569] LustreError: 84424:0:(obd_mount.c:193:lustre_start_simple()) lustre-OST0000-osd setup error -28 [ 1252.684587] LustreError: 84424:0:(tgt_mount.c:2447:server_fill_super()) Unable to start osd on /dev/mapper/ost1_flakey: -28 [ 1252.687143] LustreError: 84424:0:(super25.c:199:lustre_fill_super()) llite: Unable to mount : rc = -28 [ 1258.681372] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_hostid [ 1261.782433] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing load_modules_local [ 1265.240176] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1267.460740] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1269.535392] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1271.640515] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1273.913275] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1273.933845] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1274.015517] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 1274.027056] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1274.064476] Lustre: lustre-MDT0000: new disk, initializing [ 1274.088304] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1274.096768] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1275.424699] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 1279.631268] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1279.650632] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1279.683802] Lustre: 87261:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 1279.777086] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 1281.092085] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 1284.261366] Lustre: DEBUG MARKER: oleg429-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1285.427928] Lustre: DEBUG MARKER: oleg429-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 1287.634573] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1287.659414] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1289.591570] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 1289.625462] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 1289.667394] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 1292.685696] Lustre: DEBUG MARKER: oleg429-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1294.080572] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 1294.154594] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 1295.339116] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 1295.402716] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 1299.938620] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1299.939619] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 1299.942393] Lustre: Skipped 2 previous similar messages [ 1299.944606] LustreError: Skipped 1 previous similar message [ 1299.945297] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1299.948626] Lustre: Skipped 8 previous similar messages [ 1302.061929] LustreError: 90201:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 1302.063520] LustreError: 90201:0:(obd_class.h:479:obd_check_dev()) Skipped 17 previous similar messages [ 1302.095649] Lustre: server umount lustre-OST0000 complete [ 1309.467066] Lustre: server umount lustre-MDT0000 complete [ 1310.176876] LustreError: 87272:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1310.184043] LustreError: 87272:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 1310.692115] LustreError: 87252:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1767035803 with bad export cookie 11634147280590986381 [ 1310.694135] LustreError: MGC192.168.204.129@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1310.695194] LustreError: 87252:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 1310.806312] Lustre: server umount lustre-MDT0001 complete [ 1313.647740] Lustre: DEBUG MARKER: == conf-sanity test 84: check recovery_hard_time ========= 14:16:45 (1767035805) [ 1315.917668] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1317.416825] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 1320.370371] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1321.844601] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 1323.659597] Lustre: DEBUG MARKER: oleg429-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1324.789683] Lustre: DEBUG MARKER: oleg429-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 1326.994528] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1328.955828] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 1330.648519] Lustre: DEBUG MARKER: oleg429-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1332.007697] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 1342.281623] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 10 sec [ 1343.668199] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 1343.734421] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 1345.969977] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1345.998807] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1346.053268] Lustre: lustre-OST0001: new disk, initializing [ 1346.054408] Lustre: Skipped 2 previous similar messages [ 1346.056275] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 1346.058783] Lustre: Skipped 3 previous similar messages [ 1347.968150] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 1351.050698] Lustre: DEBUG MARKER: oleg429-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid [ 1352.472676] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid 50 [ 1355.773182] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 1355.775734] Lustre: Skipped 2 previous similar messages [ 1355.777611] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 1355.801203] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 1356.613142] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid in FULL state after 4 sec [ 1357.983607] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [ 1358.061507] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 1368.309186] Lustre: DEBUG MARKER: mds1 REPLAY BARRIER on lustre-MDT0000 [ 1375.895506] Lustre: Failing over lustre-MDT0000 [ 1376.228163] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1376.231375] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1376.232715] Lustre: Skipped 5 previous similar messages [ 1376.235463] Lustre: Skipped 4 previous similar messages [ 1376.282739] LustreError: 96258:0:(obd_class.h:479:obd_check_dev()) Device 28 not setup [ 1376.285570] LustreError: 96258:0:(obd_class.h:479:obd_check_dev()) Skipped 17 previous similar messages [ 1376.319033] Lustre: server umount lustre-MDT0000 complete [ 1380.106819] LustreError: 91269:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 192.168.204.29@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1380.112721] LustreError: 91269:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 1390.986839] LDISKFS-fs (dm-0): 4 truncates cleaned up [ 1390.988026] LDISKFS-fs (dm-0): recovery complete [ 1390.990688] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1391.026145] LustreError: MGC192.168.204.129@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1391.166713] Lustre: lustre-MDT0000: in recovery but waiting for the first client to connect [ 1392.446716] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 1395.457778] Lustre: lustre-MDT0000: Will be in recovery for at least 1:00, or until 3 clients reconnect [ 1395.458066] Lustre: 95583:0:(ldlm_lib.c:2068:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 1396.198171] Lustre: 91293:0:(ldlm_lib.c:2068:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 1396.203167] Lustre: 91293:0:(ldlm_lib.c:2068:extend_recovery_timer()) Skipped 1 previous similar message [ 1396.210262] Lustre: lustre-MDT0000-lwp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 1396.228844] LustreError: 97004:0:(ldlm_lib.c:2681:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 1396.544076] LustreError: 97004:0:(ldlm_lib.c:2681:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 1396.858886] LustreError: 97004:0:(ldlm_lib.c:2681:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 1396.864040] LustreError: 97004:0:(ldlm_lib.c:2681:replay_request_or_update()) Skipped 1 previous similar message [ 1397.176127] LustreError: 97004:0:(ldlm_lib.c:2681:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 1397.179449] LustreError: 97004:0:(ldlm_lib.c:2681:replay_request_or_update()) Skipped 1 previous similar message [ 1397.498480] Lustre: 97004:0:(ldlm_lib.c:2068:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 1397.503045] Lustre: 97004:0:(ldlm_lib.c:2068:extend_recovery_timer()) Skipped 8 previous similar messages [ 1398.130415] LustreError: 97004:0:(ldlm_lib.c:2681:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 1398.133951] LustreError: 97004:0:(ldlm_lib.c:2681:replay_request_or_update()) Skipped 3 previous similar messages [ 1398.448095] LustreError: 97004:0:(ldlm_lib.c:2681:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 1398.450287] LustreError: 97004:0:(ldlm_lib.c:2681:replay_request_or_update()) Skipped 3 previous similar messages [ 1399.705100] Lustre: 97004:0:(ldlm_lib.c:2068:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 1399.708925] Lustre: 97004:0:(ldlm_lib.c:2068:extend_recovery_timer()) Skipped 6 previous similar messages [ 1400.336950] LustreError: 97004:0:(ldlm_lib.c:2681:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 1400.340467] LustreError: 97004:0:(ldlm_lib.c:2681:replay_request_or_update()) Skipped 6 previous similar messages [ 1400.656149] LustreError: 97004:0:(ldlm_lib.c:2681:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 1400.659213] LustreError: 97004:0:(ldlm_lib.c:2681:replay_request_or_update()) Skipped 6 previous similar messages [ 1403.786019] Lustre: 97004:0:(ldlm_lib.c:2068:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 1403.789618] Lustre: 97004:0:(ldlm_lib.c:2068:extend_recovery_timer()) Skipped 12 previous similar messages [ 1404.417961] LustreError: 97004:0:(ldlm_lib.c:2681:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 1404.420780] LustreError: 97004:0:(ldlm_lib.c:2681:replay_request_or_update()) Skipped 12 previous similar messages [ 1404.736128] LustreError: 97004:0:(ldlm_lib.c:2681:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 1404.738025] LustreError: 97004:0:(ldlm_lib.c:2681:replay_request_or_update()) Skipped 12 previous similar messages [ 1411.923365] Lustre: 97004:0:(ldlm_lib.c:2068:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 1411.928281] Lustre: 97004:0:(ldlm_lib.c:2068:extend_recovery_timer()) Skipped 25 previous similar messages [ 1412.562357] LustreError: 97004:0:(ldlm_lib.c:2681:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 1412.565693] LustreError: 97004:0:(ldlm_lib.c:2681:replay_request_or_update()) Skipped 25 previous similar messages [ 1412.880080] LustreError: 97004:0:(ldlm_lib.c:2681:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 1412.882066] LustreError: 97004:0:(ldlm_lib.c:2681:replay_request_or_update()) Skipped 25 previous similar messages [ 1428.178122] Lustre: 97004:0:(ldlm_lib.c:2068:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 1428.182016] Lustre: 97004:0:(ldlm_lib.c:2068:extend_recovery_timer()) Skipped 51 previous similar messages [ 1428.810694] LustreError: 97004:0:(ldlm_lib.c:2681:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 1428.814254] LustreError: 97004:0:(ldlm_lib.c:2681:replay_request_or_update()) Skipped 51 previous similar messages [ 1429.128160] LustreError: 97004:0:(ldlm_lib.c:2681:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 1429.131291] LustreError: 97004:0:(ldlm_lib.c:2681:replay_request_or_update()) Skipped 51 previous similar messages [ 1455.032208] Lustre: 97004:0:(ldlm_lib.c:2377:target_recovery_overseer()) lustre-MDT0000 recovery is aborted by hard timeout [ 1455.035813] Lustre: 97004:0:(ldlm_lib.c:2387:target_recovery_overseer()) recovery is aborted, evict exports in recovery [ 1455.038492] Lustre: 97004:0:(genops.c:1618:class_disconnect_stale_exports()) lustre-MDT0000: disconnect stale client e38da333-126c-490f-82a4-c35d92e22498@192.168.204.29@tcp [ 1455.042804] Lustre: lustre-MDT0000: disconnecting 1 stale clients [ 1455.045103] Lustre: 97004:0:(ldlm_lib.c:1897:abort_req_replay_queue()) @@@ aborted: req@000000000853667d x1852871015470080/t0(8589934785) o101->e38da333-126c-490f-82a4-c35d92e22498@192.168.204.29@tcp:738/0 lens 592/0 e 0 to 0 dl 1767035958 ref 1 fl Complete:/604/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:0 [ 1455.055211] Lustre: lustre-MDT0000-osd: cancel update llog [0x200000400:0x1:0x0] [ 1455.058758] Lustre: lustre-MDT0000: Denying connection for new client e38da333-126c-490f-82a4-c35d92e22498 (at 192.168.204.29@tcp), waiting for 3 known clients (2 recovered, 0 in progress, and 1 evicted) to recover in 0:00 [ 1455.064623] Lustre: lustre-MDT0001-osp-MDT0000: cancel update llog [0x240000401:0x1:0x0] [ 1455.081293] Lustre: lustre-MDT0000-osp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 1455.084950] Lustre: Skipped 2 previous similar messages [ 1455.089790] Lustre: lustre-MDT0000: Recovery over after 1:00, of 3 clients 2 recovered and 1 was evicted. [ 1455.111493] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:81 to 0x280000401:577) [ 1455.111636] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:49 to 0x2c0000401:545) [ 1460.707203] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1460.709744] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1460.714655] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1460.717426] Lustre: Skipped 1 previous similar message [ 1466.619332] Lustre: server umount lustre-OST0000 complete [ 1467.877283] LustreError: 92871:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1467.880836] LustreError: 92871:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 13 previous similar messages [ 1482.208177] Lustre: lustre-OST0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 1482.273331] Lustre: server umount lustre-OST0001 complete [ 1485.792891] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1485.796769] LustreError: Skipped 2 previous similar messages [ 1489.703763] Lustre: server umount lustre-MDT0000 complete [ 1490.977441] LustreError: 91256:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1767035983 with bad export cookie 11634147280591037565 [ 1490.980563] LustreError: MGC192.168.204.129@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1490.981439] LustreError: 91256:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1491.097960] Lustre: server umount lustre-MDT0001 complete [ 1494.019284] Lustre: DEBUG MARKER: == conf-sanity test 85: osd_ost init: fail ea_fid_set ==== 14:19:46 (1767035986) [ 1496.532728] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1496.535496] Lustre: *** cfs_fail_loc=197, val=0*** [ 1511.968531] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 1517.088307] LustreError: 98959:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.204.129@tcp: failed processing log, type 4: rc = -110 [ 1522.218263] Lustre: *** cfs_fail_loc=197, val=0*** [ 1522.220195] Lustre: Skipped 65 previous similar messages [ 1542.688420] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1542.690218] Lustre: Skipped 7 previous similar messages [ 1544.609061] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 1546.427240] Lustre: DEBUG MARKER: oleg429-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1547.112858] LustreError: 99623:0:(obd_class.h:479:obd_check_dev()) Device 1 not setup [ 1547.115177] LustreError: 99623:0:(obd_class.h:479:obd_check_dev()) Skipped 33 previous similar messages [ 1547.154831] Lustre: server umount lustre-OST0000 complete [ 1550.216801] Lustre: DEBUG MARKER: == conf-sanity test 86: Replacing mkfs.lustre -G option == 14:20:42 (1767036042) [ 1564.530850] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1564.709986] LustreError: 100467:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1564.718197] LustreError: 100467:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 4 previous similar messages [ 1566.150261] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 1569.335034] Lustre: DEBUG MARKER: oleg429-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1571.371070] LustreError: 100498:0:(osp_object.c:617:osp_attr_get()) lustre-MDT0001-osp-MDT0000: osp_attr_get update error [0x200000009:0x1:0x0]: rc = -5 [ 1571.375658] LustreError: 100498:0:(lod_sub_object.c:917:lod_sub_prep_llog()) lustre-MDT0000-mdtlov: can't get id from catalogs: rc = -5 [ 1571.379329] LustreError: 100498:0:(lod_dev.c:510:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 7, retries 0, failed: rc = -5 [ 1571.516302] Lustre: server umount lustre-MDT0000 complete [ 1577.483520] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1583.563721] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_hostid [ 1586.854450] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing load_modules_local [ 1590.095638] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1592.332459] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1594.378262] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1596.933849] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1599.745210] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1599.779203] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1599.887162] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 1599.903333] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1599.951971] Lustre: lustre-MDT0000: new disk, initializing [ 1600.002487] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1601.513173] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 1606.429706] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1606.466306] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1606.507396] Lustre: 104532:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 1606.620105] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 1608.010687] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 1611.221697] Lustre: DEBUG MARKER: oleg429-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1612.465157] Lustre: DEBUG MARKER: oleg429-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 1614.861308] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1614.890975] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1615.923840] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 1617.192440] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 1620.673523] Lustre: DEBUG MARKER: oleg429-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1622.263706] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 1622.339813] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 1623.921496] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 1624.012088] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 1626.082166] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1626.083729] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1626.085639] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1626.088918] Lustre: Skipped 6 previous similar messages [ 1626.090899] Lustre: Skipped 11 previous similar messages [ 1638.880152] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 1638.932483] Lustre: server umount lustre-OST0000 complete [ 1646.294581] Lustre: server umount lustre-MDT0000 complete [ 1647.577541] LustreError: 106282:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1767036140 with bad export cookie 11634147280591048289 [ 1647.579486] LustreError: MGC192.168.204.129@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1647.581835] LustreError: 106282:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 1647.712523] Lustre: server umount lustre-MDT0001 complete [ 1650.626153] Lustre: DEBUG MARKER: == conf-sanity test 87: check if MDT inode can hold EAs with N stripes properly ========================================================== 14:22:22 (1767036142) [ 1653.550899] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing unload_modules_local [ 1654.649301] Key type lgssc unregistered [ 1654.795385] LNet: 108602:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1654.798208] LNetError: 108602:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1654.805385] LNet: Removed LNI 192.168.204.129@tcp [ 1655.143639] Key type .llcrypt unregistered [ 1655.145505] Key type ._llcrypt unregistered [ 1661.270058] Key type ._llcrypt registered [ 1661.271265] Key type .llcrypt registered [ 1661.310060] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_hostid [ 1667.339484] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing load_modules_local [ 1667.628618] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1667.665030] alg: No test for adler32 (adler32-zlib) [ 1668.516628] Lustre: Lustre: Build Version: 2.17.0_RC3_1_ga0d6154 [ 1668.593678] LNet: Added LNI 192.168.204.129@tcp [8/256/0/180] [ 1670.168120] Key type lgssc registered [ 1670.496895] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1673.255552] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1675.335205] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1677.363395] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1679.476180] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1681.287202] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1683.448054] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1683.468571] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1683.472339] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1684.550144] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 1684.561794] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1684.601961] Lustre: lustre-MDT0000: new disk, initializing [ 1684.627373] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1684.633642] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1685.910300] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 1689.879606] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1689.901771] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1689.996743] Lustre: lustre-OST0000: new disk, initializing [ 1689.998446] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1690.000273] Lustre: Skipped 1 previous similar message [ 1690.022710] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1691.772118] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 1691.774976] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 1691.782340] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 1691.874265] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 1695.853314] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1695.874636] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1695.919385] Lustre: lustre-OST0001: new disk, initializing [ 1695.920863] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 1695.943914] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 1696.967408] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:1:ost [ 1696.971733] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:1:ost] [ 1696.981696] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x280000400 [ 1697.847697] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 1702.370328] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1702.371293] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1702.376124] Lustre: Skipped 1 previous similar message [ 1707.489501] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1707.491251] Lustre: Skipped 2 previous similar messages [ 1707.901036] LustreError: 114155:0:(obd_class.h:479:obd_check_dev()) Device 17 not setup [ 1707.951645] Lustre: server umount lustre-MDT0000 complete [ 1710.084301] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: (null) [ 1711.604515] LustreError: 111911:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1767036204 with bad export cookie 4217380629969476313 [ 1711.607576] LustreError: MGC192.168.204.129@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1711.608747] LustreError: 111911:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 1711.628075] LustreError: 114749:0:(obd_class.h:479:obd_check_dev()) Device 12 not setup [ 1711.630564] LustreError: 114749:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 1711.649637] Lustre: server umount lustre-OST0000 complete [ 1712.878644] LustreError: 114949:0:(obd_class.h:479:obd_check_dev()) Device 2 not setup [ 1712.880571] LustreError: 114949:0:(obd_class.h:479:obd_check_dev()) Skipped 1 previous similar message [ 1712.921106] Lustre: server umount lustre-OST0001 complete [ 1715.773328] Lustre: DEBUG MARKER: == conf-sanity test 88: check the default mount options can be overridden ========================================================== 14:23:27 (1767036207) [ 1718.790754] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing load_modules_local [ 1721.625988] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1721.944674] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1723.598768] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1723.948438] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1726.089492] Lustre: DEBUG MARKER: == conf-sanity test 89: check tunefs --param and --erase-param{s} options ========================================================== 14:23:38 (1767036218) [ 1730.215427] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1730.867675] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1731.536758] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1732.159664] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1732.469793] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1733.127582] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1738.815367] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_hostid [ 1741.965102] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing load_modules_local [ 1745.398667] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1747.625151] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1749.618853] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1751.727055] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1753.971392] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1753.995735] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1754.078259] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 1754.091348] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1754.128924] Lustre: lustre-MDT0000: new disk, initializing [ 1754.158336] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1754.166170] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1755.440819] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 1759.504066] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1759.526598] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1759.553693] Lustre: 120260:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 1759.583757] Lustre: lustre-MDT0001: Not available for connect from 0@lo (not set up) [ 1759.584809] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 1759.589434] Lustre: Skipped 1 previous similar message [ 1759.623979] Lustre: lustre-MDT0001: new disk, initializing [ 1759.653537] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 1759.665334] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 1759.668462] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 1760.977748] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 1764.022449] Lustre: DEBUG MARKER: oleg429-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1765.185378] Lustre: DEBUG MARKER: oleg429-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 1767.394491] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1767.419530] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1768.891566] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 1768.895309] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 1768.933978] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 1769.480661] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 1772.631315] Lustre: DEBUG MARKER: oleg429-client.virtnet: executing wait_import_state_mount (FULL|IDLE) failover.node=192.0.2.254@tcp0,osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1774.101384] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 1774.177352] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 1775.471349] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 1775.546771] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 1779.169403] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 1779.170336] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1779.171634] LustreError: Skipped 1 previous similar message [ 1779.176320] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1779.178279] Lustre: Skipped 1 previous similar message [ 1782.255102] LustreError: 123201:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 1782.257230] LustreError: 123201:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 1782.296858] Lustre: server umount lustre-OST0000 complete [ 1784.289670] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1784.290428] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1784.293354] Lustre: Skipped 2 previous similar messages [ 1784.296536] Lustre: Skipped 1 previous similar message [ 1789.701259] LustreError: 123402:0:(obd_class.h:479:obd_check_dev()) Device 22 not setup [ 1789.703394] LustreError: 123402:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 1789.732131] Lustre: server umount lustre-MDT0000 complete [ 1791.006250] LustreError: 120252:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1767036283 with bad export cookie 4217380629969477524 [ 1791.009170] LustreError: MGC192.168.204.129@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1791.010596] LustreError: 120252:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1791.118460] Lustre: server umount lustre-MDT0001 complete [ 1793.880926] Lustre: DEBUG MARKER: == conf-sanity test 90a: check max_mod_rpcs_in_flight is enforced ========================================================== 14:24:46 (1767036286) [ 1796.075970] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1796.228827] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1796.231993] Lustre: Skipped 1 previous similar message [ 1797.450576] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 1800.142846] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1801.559351] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 1803.250744] Lustre: DEBUG MARKER: oleg429-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1804.374708] Lustre: DEBUG MARKER: oleg429-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 1806.674080] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1808.586626] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 1810.237628] Lustre: DEBUG MARKER: oleg429-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1828.141340] Lustre: *** cfs_fail_loc=159, val=0*** [ 1843.467148] Lustre: lustre-MDT0000: Client 3df004d8-a25f-4d74-a8f3-9793c1726ce9 (at 192.168.204.29@tcp) reconnecting [ 1843.995878] Lustre: *** cfs_fail_loc=159, val=0*** [ 1843.997715] Lustre: Skipped 5 previous similar messages [ 1859.346200] Lustre: lustre-MDT0000: Client 3df004d8-a25f-4d74-a8f3-9793c1726ce9 (at 192.168.204.29@tcp) reconnecting [ 1862.115441] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1862.115935] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1862.118765] LustreError: Skipped 1 previous similar message [ 1862.124780] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1862.127238] Lustre: Skipped 3 previous similar messages [ 1866.669080] LustreError: 126781:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 1866.671564] LustreError: 126781:0:(obd_class.h:479:obd_check_dev()) Skipped 13 previous similar messages [ 1866.710435] Lustre: server umount lustre-OST0000 complete [ 1871.840731] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1871.842516] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1871.846740] Lustre: Skipped 1 previous similar message [ 1874.145333] Lustre: server umount lustre-MDT0000 complete [ 1875.547449] LustreError: 124254:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1767036368 with bad export cookie 4217380629969478378 [ 1875.550809] LustreError: MGC192.168.204.129@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1875.551932] LustreError: 124254:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1875.684595] Lustre: server umount lustre-MDT0001 complete [ 1879.566612] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing unload_modules_local [ 1880.828461] Key type lgssc unregistered [ 1880.974979] LNet: 127715:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1880.977918] LNetError: 127715:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1880.985314] LNet: Removed LNI 192.168.204.129@tcp [ 1881.317939] Key type .llcrypt unregistered [ 1881.319038] Key type ._llcrypt unregistered [ 1890.968696] Key type ._llcrypt registered [ 1890.969474] Key type .llcrypt registered [ 1891.001544] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing load_modules_local [ 1891.383612] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1891.395488] alg: No test for adler32 (adler32-zlib) [ 1892.239964] Lustre: Lustre: Build Version: 2.17.0_RC3_1_ga0d6154 [ 1892.319775] LNet: Added LNI 192.168.204.129@tcp [8/256/0/180] [ 1893.896120] Key type lgssc registered [ 1894.245264] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1896.750886] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1896.757088] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1897.907143] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1899.199473] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 1902.098039] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1902.237838] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 1903.597883] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 1905.467953] Lustre: DEBUG MARKER: oleg429-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1906.641620] Lustre: DEBUG MARKER: oleg429-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 1908.905169] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1909.032938] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1910.859156] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 1912.472820] Lustre: DEBUG MARKER: oleg429-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1922.346787] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:42 to 0x280000401:65) [ 1927.330607] Lustre: *** cfs_fail_loc=159, val=0*** [ 1943.817125] Lustre: lustre-MDT0000: Client 83495541-0620-4c57-a678-632fdbfe7e14 (at 192.168.204.29@tcp) reconnecting [ 1944.418700] Lustre: *** cfs_fail_loc=159, val=0*** [ 1960.717553] Lustre: lustre-MDT0001: Client 83495541-0620-4c57-a678-632fdbfe7e14 (at 192.168.204.29@tcp) reconnecting [ 1961.188646] Lustre: *** cfs_fail_loc=159, val=0*** [ 1961.190339] Lustre: Skipped 3 previous similar messages [ 1976.576689] Lustre: lustre-MDT0001: Client 83495541-0620-4c57-a678-632fdbfe7e14 (at 192.168.204.29@tcp) reconnecting [ 1992.096755] Lustre: 131832:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 1992.922684] Lustre: *** cfs_fail_loc=159, val=0*** [ 1992.924965] Lustre: Skipped 4 previous similar messages [ 2008.334061] Lustre: lustre-MDT0000: Client d082737d-56d3-41b5-bc1d-4a28daa48ad3 (at 192.168.204.29@tcp) reconnecting [ 2008.999291] Lustre: *** cfs_fail_loc=159, val=0*** [ 2009.001018] Lustre: Skipped 13 previous similar messages [ 2025.229954] Lustre: lustre-MDT0000: Client d082737d-56d3-41b5-bc1d-4a28daa48ad3 (at 192.168.204.29@tcp) reconnecting [ 2025.750969] Lustre: 132124:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 2030.049184] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2030.050556] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2030.051911] LustreError: Skipped 1 previous similar message [ 2030.056593] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2030.059966] Lustre: Skipped 1 previous similar message [ 2033.071073] LustreError: 132222:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 2033.101220] Lustre: server umount lustre-OST0000 complete [ 2035.169597] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2035.173823] Lustre: Skipped 1 previous similar message [ 2035.175968] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2035.177654] Lustre: Skipped 1 previous similar message [ 2040.289479] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2040.291531] Lustre: Skipped 1 previous similar message [ 2040.512453] LustreError: 132424:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup [ 2040.514333] LustreError: 132424:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 2040.546864] Lustre: server umount lustre-MDT0000 complete [ 2041.744174] LustreError: 131952:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1767036534 with bad export cookie 6930096472980126033 [ 2041.747634] LustreError: MGC192.168.204.129@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2041.750624] LustreError: 131952:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2041.813197] LustreError: 132625:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [ 2041.815101] LustreError: 132625:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 2041.863489] Lustre: server umount lustre-MDT0001 complete [ 2045.157735] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing unload_modules_local [ 2046.181768] Key type lgssc unregistered [ 2046.314367] LNet: 133155:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2046.317655] LNetError: 133155:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2046.325262] LNet: Removed LNI 192.168.204.129@tcp [ 2046.600075] Key type .llcrypt unregistered [ 2046.601267] Key type ._llcrypt unregistered [ 2055.142451] Key type ._llcrypt registered [ 2055.144246] Key type .llcrypt registered [ 2055.185167] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing load_modules_local [ 2055.530807] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2055.577416] alg: No test for adler32 (adler32-zlib) [ 2056.431034] Lustre: Lustre: Build Version: 2.17.0_RC3_1_ga0d6154 [ 2056.507791] LNet: Added LNI 192.168.204.129@tcp [8/256/0/180] [ 2058.080165] Key type lgssc registered [ 2058.429271] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2060.912774] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 2060.919726] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2062.068935] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2063.273433] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 2066.000726] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2066.119252] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2067.361519] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 2069.100140] Lustre: DEBUG MARKER: oleg429-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2070.256395] Lustre: DEBUG MARKER: oleg429-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2072.482342] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2072.587816] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2074.365154] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 2076.023782] Lustre: DEBUG MARKER: oleg429-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2076.711229] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:97) [ 2086.950630] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:129) [ 2093.949813] Lustre: 136801:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 2094.593068] Lustre: 136849:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 2095.080916] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 in log params [ 2096.192973] Lustre: 136995:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre/mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 (mode = 0) failed: rc = -17 [ 2096.493233] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=7 in log params [ 2096.496088] Lustre: Skipped 1 previous similar message [ 2096.784472] Lustre: 137093:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 2100.194958] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2100.195780] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2100.199071] Lustre: Skipped 1 previous similar message [ 2100.204215] Lustre: Skipped 1 previous similar message [ 2103.790232] LustreError: 137191:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 2103.825116] Lustre: server umount lustre-OST0000 complete [ 2105.314001] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2105.319551] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2105.321496] Lustre: Skipped 1 previous similar message [ 2110.433073] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2110.435292] Lustre: Skipped 1 previous similar message [ 2111.229771] LustreError: 137400:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup [ 2111.231868] LustreError: 137400:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 2111.258500] Lustre: server umount lustre-MDT0000 complete [ 2112.465992] LustreError: 136898:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1767036605 with bad export cookie 16401095378515069827 [ 2112.470493] LustreError: MGC192.168.204.129@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2112.470750] LustreError: 136898:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2112.543091] LustreError: 137602:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [ 2112.545183] LustreError: 137602:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 2112.599481] Lustre: server umount lustre-MDT0001 complete [ 2116.108846] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing unload_modules_local [ 2117.097325] Key type lgssc unregistered [ 2117.219340] LNet: 138134:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2117.222255] LNetError: 138134:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2117.233277] LNet: Removed LNI 192.168.204.129@tcp [ 2117.513618] Key type .llcrypt unregistered [ 2117.514938] Key type ._llcrypt unregistered [ 2126.501228] Key type ._llcrypt registered [ 2126.502120] Key type .llcrypt registered [ 2126.538767] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing load_modules_local [ 2126.924445] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2126.932726] alg: No test for adler32 (adler32-zlib) [ 2127.785837] Lustre: Lustre: Build Version: 2.17.0_RC3_1_ga0d6154 [ 2127.861353] LNet: Added LNI 192.168.204.129@tcp [8/256/0/180] [ 2129.440144] Key type lgssc registered [ 2129.785917] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2132.302517] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 2132.308910] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2133.462305] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2134.740792] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 2137.458164] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2137.580617] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2138.777343] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 2140.576387] Lustre: DEBUG MARKER: oleg429-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2141.751912] Lustre: DEBUG MARKER: oleg429-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2143.893046] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2144.013456] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2145.852230] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 2147.110661] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:129) [ 2147.569043] Lustre: DEBUG MARKER: oleg429-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2158.374068] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:161) [ 2165.664718] Lustre: *** cfs_fail_loc=159, val=0*** [ 2165.666531] Lustre: Skipped 1 previous similar message [ 2181.903208] Lustre: lustre-MDT0000: Client c4499450-b20d-4f37-b13f-60ae73d12b31 (at 192.168.204.29@tcp) reconnecting [ 2198.496889] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2198.499421] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2198.505782] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2199.521377] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2199.521591] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2199.522605] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2199.527089] Lustre: Skipped 1 previous similar message [ 2202.732084] LustreError: 141908:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 2202.764752] Lustre: server umount lustre-OST0000 complete [ 2204.128744] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2204.130595] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2204.133894] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2209.761030] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2209.763313] Lustre: Skipped 2 previous similar messages [ 2210.109193] LustreError: 142110:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup [ 2210.112153] LustreError: 142110:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 2210.137945] Lustre: server umount lustre-MDT0000 complete [ 2211.270210] LustreError: 139486:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1767036704 with bad export cookie 15453904053734135106 [ 2211.273254] LustreError: 139486:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 2211.273637] LustreError: MGC192.168.204.129@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2211.339274] LustreError: 142311:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [ 2211.340794] LustreError: 142311:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 2211.385203] Lustre: server umount lustre-MDT0001 complete [ 2214.886200] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing unload_modules_local [ 2215.822934] Key type lgssc unregistered [ 2215.952294] LNet: 142842:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2215.954322] LNetError: 142842:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2215.962265] LNet: Removed LNI 192.168.204.129@tcp [ 2216.230103] Key type .llcrypt unregistered [ 2216.231030] Key type ._llcrypt unregistered [ 2225.197480] Key type ._llcrypt registered [ 2225.198312] Key type .llcrypt registered [ 2225.228882] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing load_modules_local [ 2225.604870] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2225.635083] alg: No test for adler32 (adler32-zlib) [ 2226.496634] Lustre: Lustre: Build Version: 2.17.0_RC3_1_ga0d6154 [ 2226.583481] LNet: Added LNI 192.168.204.129@tcp [8/256/0/180] [ 2228.168229] Key type lgssc registered [ 2228.506506] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2230.934213] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 2230.941077] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2232.091742] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2233.360718] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 2236.124749] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2236.251428] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2237.520919] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 2239.211819] Lustre: DEBUG MARKER: oleg429-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2240.333763] Lustre: DEBUG MARKER: oleg429-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2242.564441] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2242.677307] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2244.421043] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 2246.010191] Lustre: DEBUG MARKER: oleg429-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2246.757245] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:161) [ 2256.998765] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:170 to 0x280000401:193) [ 2274.598718] Lustre: lustre-OST0000: evicting d3fc4177-d700-4593-863c-59bc8abd0c63 (at 192.168.204.29@tcp) by administrative request [ 2274.601714] Lustre: Skipped 1 previous similar message [ 2303.459570] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2303.460068] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2303.461174] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2303.461395] LustreError: Skipped 1 previous similar message [ 2303.466429] Lustre: Skipped 1 previous similar message [ 2308.577624] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2308.579469] Lustre: Skipped 2 previous similar messages [ 2313.184102] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2313.201092] LustreError: 147023:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 2313.234294] Lustre: server umount lustre-OST0000 complete [ 2318.304914] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2318.306972] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2318.310786] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2320.639959] LustreError: 147227:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup [ 2320.642053] LustreError: 147227:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 2320.672238] Lustre: server umount lustre-MDT0000 complete [ 2322.004191] LustreError: 147229:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1767036814 with bad export cookie 10987838663415613877 [ 2322.007774] LustreError: MGC192.168.204.129@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2322.008873] LustreError: 147229:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2322.088267] LustreError: 147430:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [ 2322.090017] LustreError: 147430:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 2322.141079] Lustre: server umount lustre-MDT0001 complete [ 2325.903833] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing unload_modules_local [ 2326.943596] Key type lgssc unregistered [ 2327.090513] LNet: 147960:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2327.094256] LNetError: 147960:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2327.101384] LNet: Removed LNI 192.168.204.129@tcp [ 2327.427437] Key type .llcrypt unregistered [ 2327.428397] Key type ._llcrypt unregistered [ 2351.886211] Key type ._llcrypt registered [ 2351.887113] Key type .llcrypt registered [ 2351.930350] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing load_modules_local [ 2352.305522] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2352.335379] alg: No test for adler32 (adler32-zlib) [ 2353.200018] Lustre: Lustre: Build Version: 2.17.0_RC3_1_ga0d6154 [ 2353.290337] LNet: Added LNI 192.168.204.129@tcp [8/256/0/180] [ 2354.880201] Key type lgssc registered [ 2355.217423] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2357.726522] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 2357.731971] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2358.895658] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2360.241220] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 2363.162916] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2363.301499] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2364.734028] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 2366.670291] Lustre: DEBUG MARKER: oleg429-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2367.924987] Lustre: DEBUG MARKER: oleg429-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2370.244250] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2370.376461] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2372.300533] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 2373.481251] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:193) [ 2374.259301] Lustre: DEBUG MARKER: oleg429-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2383.719791] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:195 to 0x280000401:225) [ 2388.963588] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2388.964158] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2388.966085] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2388.968129] Lustre: Skipped 1 previous similar message [ 2388.972935] LustreError: Skipped 1 previous similar message [ 2393.456185] LustreError: 152677:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 2393.495592] Lustre: server umount lustre-OST0000 complete [ 2399.201690] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2399.202099] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2399.202751] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2399.202755] Lustre: Skipped 1 previous similar message [ 2399.209377] Lustre: Skipped 1 previous similar message [ 2400.829222] LustreError: 152879:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup [ 2400.830786] LustreError: 152879:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 2400.853573] Lustre: server umount lustre-MDT0000 complete [ 2402.089444] LustreError: 150383:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1767036895 with bad export cookie 3275228665306610910 [ 2402.093327] LustreError: MGC192.168.204.129@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2402.094373] LustreError: 150383:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2402.162522] LustreError: 153081:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [ 2402.164119] LustreError: 153081:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 2402.211388] Lustre: server umount lustre-MDT0001 complete [ 2406.030520] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing unload_modules_local [ 2407.122451] Key type lgssc unregistered [ 2407.263431] LNet: 153612:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2407.266630] LNetError: 153612:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2407.277363] LNet: Removed LNI 192.168.204.129@tcp [ 2407.592028] Key type .llcrypt unregistered [ 2407.593725] Key type ._llcrypt unregistered [ 2416.950212] Key type ._llcrypt registered [ 2416.951229] Key type .llcrypt registered [ 2416.993179] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing load_modules_local [ 2417.398934] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2417.427316] alg: No test for adler32 (adler32-zlib) [ 2418.286801] Lustre: Lustre: Build Version: 2.17.0_RC3_1_ga0d6154 [ 2418.397595] LNet: Added LNI 192.168.204.129@tcp [8/256/0/180] [ 2419.992203] Key type lgssc registered [ 2420.382847] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2422.811307] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2425.317287] Lustre: DEBUG MARKER: == conf-sanity test 100: check lshowmount lists MGS, MDT, OST and 0@lo ========================================================== 14:35:17 (1767036917) [ 2430.491689] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_hostid [ 2433.704189] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing load_modules_local [ 2437.074437] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2439.283960] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2441.315752] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2443.489379] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2445.719347] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2445.736324] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 2445.740287] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2446.821903] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2446.832222] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2446.871704] Lustre: lustre-MDT0000: new disk, initializing [ 2446.898306] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2446.907060] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2448.307261] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 2452.451945] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2452.474225] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2452.514718] Lustre: 157804:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2452.538729] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 2452.541501] Lustre: Skipped 1 previous similar message [ 2452.580440] Lustre: lustre-MDT0001: new disk, initializing [ 2452.621590] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2452.636266] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 2452.640723] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2453.966217] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 2457.069854] Lustre: DEBUG MARKER: oleg429-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2458.302099] Lustre: DEBUG MARKER: oleg429-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2460.588926] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2460.615078] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2460.719710] Lustre: lustre-OST0000: new disk, initializing [ 2460.723044] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 2460.754988] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2462.667320] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 2465.824351] Lustre: DEBUG MARKER: oleg429-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2466.298195] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 2466.301544] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2466.334334] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2476.515209] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2476.516532] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2476.518286] LustreError: Skipped 1 previous similar message [ 2476.523477] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2476.526836] Lustre: Skipped 1 previous similar message [ 2480.238102] LustreError: 160392:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 2480.279855] Lustre: server umount lustre-OST0000 complete [ 2481.634405] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2481.635090] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2481.637094] Lustre: Skipped 2 previous similar messages [ 2486.753422] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2486.756453] Lustre: Skipped 2 previous similar messages [ 2487.613924] LustreError: 160594:0:(obd_class.h:479:obd_check_dev()) Device 22 not setup [ 2487.615624] LustreError: 160594:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 2487.640984] Lustre: server umount lustre-MDT0000 complete [ 2488.899333] LustreError: 158587:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1767036981 with bad export cookie 9286912672371499603 [ 2488.902475] LustreError: 158587:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 2488.903274] LustreError: MGC192.168.204.129@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2488.970390] LustreError: 160796:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 2488.973464] LustreError: 160796:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 2489.025792] Lustre: server umount lustre-MDT0001 complete [ 2492.827978] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing unload_modules_local [ 2493.864812] Key type lgssc unregistered [ 2493.991337] LNet: 161327:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2493.994530] LNetError: 161327:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2494.001305] LNet: Removed LNI 192.168.204.129@tcp [ 2494.288732] Key type .llcrypt unregistered [ 2494.289696] Key type ._llcrypt unregistered [ 2503.925165] Key type ._llcrypt registered [ 2503.926103] Key type .llcrypt registered [ 2503.970714] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing load_modules_local [ 2504.267613] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2504.290110] alg: No test for adler32 (adler32-zlib) [ 2505.158063] Lustre: Lustre: Build Version: 2.17.0_RC3_1_ga0d6154 [ 2505.245891] LNet: Added LNI 192.168.204.129@tcp [8/256/0/180] [ 2506.840151] Key type lgssc registered [ 2507.263388] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2509.900052] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 2509.905693] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2511.069247] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2512.360034] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 2515.440498] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2515.570825] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2516.995933] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 2518.844251] Lustre: DEBUG MARKER: oleg429-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2520.125503] Lustre: DEBUG MARKER: oleg429-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2522.353672] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2522.472919] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2524.469491] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 2526.327287] Lustre: DEBUG MARKER: oleg429-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2535.785709] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:3 to 0x280000401:33) [ 2539.125368] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 2539.131021] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2539.135523] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 2539.139413] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2539.142712] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 2539.477522] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2539.820601] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 2539.823136] Lustre: Skipped 1 previous similar message [ 2539.828551] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2539.831773] Lustre: Skipped 1 previous similar message [ 2539.834051] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 2539.835802] Lustre: Skipped 1 previous similar message [ 2539.836907] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2539.839698] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 2539.841155] Lustre: Skipped 1 previous similar message [ 2540.159058] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2540.463459] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2540.783230] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2541.101315] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 2541.105144] Lustre: Skipped 3 previous similar messages [ 2541.112376] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2541.117580] Lustre: Skipped 3 previous similar messages [ 2541.119242] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 2541.122185] Lustre: Skipped 3 previous similar messages [ 2541.123883] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2541.127505] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 2541.129811] Lustre: Skipped 3 previous similar messages [ 2541.455049] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2541.811305] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2542.138777] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2542.483257] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2542.803034] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2543.106890] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 2543.109529] Lustre: Skipped 5 previous similar messages [ 2543.115046] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2543.119827] Lustre: Skipped 5 previous similar messages [ 2543.122330] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 2543.125866] Lustre: Skipped 5 previous similar messages [ 2543.128353] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2543.133416] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 2543.135845] Lustre: Skipped 5 previous similar messages [ 2543.449983] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2543.761326] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2544.073091] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2544.404303] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2544.730624] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2545.046280] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2545.364867] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2545.704701] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2546.025514] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2546.339909] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2546.649491] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2546.974320] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2547.305091] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 2547.307894] Lustre: Skipped 12 previous similar messages [ 2547.313896] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2547.318885] Lustre: Skipped 12 previous similar messages [ 2547.320461] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 2547.322192] Lustre: Skipped 12 previous similar messages [ 2547.323959] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2547.327703] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 2547.329433] Lustre: Skipped 12 previous similar messages [ 2547.635382] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2547.935928] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2548.266199] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2548.597452] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2548.915277] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2549.216218] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2549.515793] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2549.815759] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2550.112195] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2550.418033] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2550.744554] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2551.066995] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2551.386303] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2551.694062] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2552.024865] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2552.318994] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2552.634412] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2552.953332] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2553.287523] LustreError: 162718:0:(osp_precreate.c:651:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 2553.291250] LustreError: 162718:0:(osp_precreate.c:1355:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 2553.292806] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2553.617284] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2553.944804] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2554.269862] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2554.618209] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2554.932971] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2555.256203] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2555.555144] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 2555.557413] Lustre: Skipped 25 previous similar messages [ 2555.563260] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2555.566447] Lustre: Skipped 25 previous similar messages [ 2555.567808] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 2555.570198] Lustre: Skipped 25 previous similar messages [ 2555.571483] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2555.575125] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 2555.577607] Lustre: Skipped 25 previous similar messages [ 2555.898290] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2556.219140] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2556.522251] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2556.831668] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2557.193923] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2557.528145] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2557.851055] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2558.178992] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2558.519222] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2558.833244] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2559.144217] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2559.455747] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2559.791500] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2560.103566] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2560.422772] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2560.735434] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2561.063427] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2561.366868] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2561.686317] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2561.997788] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2562.323095] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2562.636895] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2562.946170] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2563.247378] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2563.565709] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2563.868110] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2564.184344] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2564.485241] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2564.828101] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2565.131575] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2565.442116] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2565.748588] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2566.058920] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2566.367805] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2566.668472] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2566.984099] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2567.305424] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2567.608384] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2567.920074] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2568.227841] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2568.544186] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2568.851114] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2569.157266] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2569.461974] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2569.779525] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2570.100478] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2570.409571] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2570.726921] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2571.069370] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2571.378032] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2571.694178] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 2571.696167] Lustre: Skipped 50 previous similar messages [ 2571.701732] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2571.705779] Lustre: Skipped 50 previous similar messages [ 2571.707442] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 2571.709200] Lustre: Skipped 50 previous similar messages [ 2571.711263] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2571.714728] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 2571.716247] Lustre: Skipped 50 previous similar messages [ 2572.021947] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2572.356490] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2572.670451] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2572.980732] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2573.285418] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2573.598474] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2573.900215] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2574.215720] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2574.529974] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2574.854475] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2575.155557] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2575.450375] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2575.766584] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2576.092996] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2576.397594] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2576.701838] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2576.995776] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2577.308650] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2577.600725] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2577.892100] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2578.203159] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2578.522390] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2578.837582] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2579.168480] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2579.505698] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2579.860103] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2580.181687] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2580.493264] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2580.794855] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2581.098181] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2581.393083] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2581.669227] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2581.961866] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2582.263787] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2582.550746] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2582.838916] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2583.144053] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2583.466752] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2583.756286] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2584.056481] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2584.374895] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2584.675385] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2584.966539] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2585.256359] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2585.535922] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2585.828936] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2586.115230] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2586.406735] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2586.696585] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2586.974798] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2587.250884] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2587.517294] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2587.789426] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2588.063194] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2588.341481] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2588.611152] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2588.882632] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2589.163949] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2589.440193] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2589.750630] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2590.051298] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2590.365729] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2590.645312] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2590.926922] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2591.202791] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2591.485618] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2591.759909] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2592.039920] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2592.306094] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2592.602268] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2592.898434] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2593.200736] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2593.495887] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2593.811328] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2594.121522] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2594.428446] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2594.737550] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2595.065541] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2595.363225] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2595.674701] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2595.986883] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2596.308092] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2596.605279] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2596.903550] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2597.197031] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2597.522641] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2597.840869] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2598.164778] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2598.469212] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2598.791383] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2599.095944] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2599.405658] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2599.700591] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2600.011410] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2600.293246] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2600.578761] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2600.865616] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2601.178693] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2601.469490] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2601.769876] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2602.073342] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2602.392558] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2602.688170] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2602.990450] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2603.290854] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2603.601294] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2603.906588] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 2603.908332] Lustre: Skipped 106 previous similar messages [ 2603.913239] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2603.917028] Lustre: Skipped 106 previous similar messages [ 2603.918301] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 2603.921125] Lustre: Skipped 106 previous similar messages [ 2603.923138] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2603.926718] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 2603.928767] Lustre: Skipped 106 previous similar messages [ 2604.251760] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2604.581941] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2604.920659] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2605.227805] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2605.547482] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2605.864767] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2606.174160] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2606.456591] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2606.794702] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2607.104479] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2607.423513] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2607.735584] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2608.057868] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2608.371201] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2608.702276] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2609.012133] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2609.326426] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2609.631408] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2609.959024] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2610.282106] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2610.602315] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2610.913959] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2611.232669] LustreError: 162718:0:(osp_precreate.c:651:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 2611.235773] LustreError: 162718:0:(osp_precreate.c:1355:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 2611.237396] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2611.557352] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2611.867184] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2612.182889] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2612.492318] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2612.797276] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2613.098096] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2613.424780] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2613.762152] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2614.070233] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2614.394441] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2614.705078] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2615.045051] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2615.356191] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2615.671323] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2615.983292] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2616.310366] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2616.619444] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2616.941873] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2617.261654] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2617.592080] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2617.915314] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2618.221106] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2618.535908] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2618.862461] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2619.179558] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2619.495296] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2619.816572] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2620.144727] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2620.442255] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2620.744837] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2621.038798] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2621.360165] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2621.654202] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2621.953946] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2622.262475] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2622.595366] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2622.906629] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2623.206725] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2623.521755] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2623.843909] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2624.152617] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2624.467067] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2624.773993] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2625.095038] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2625.399657] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2625.720993] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2626.037644] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2626.348753] LustreError: 162718:0:(osp_precreate.c:651:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 2626.352250] LustreError: 162718:0:(osp_precreate.c:1355:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 2626.352891] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2626.656474] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2626.952259] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2627.241292] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2627.566035] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2627.886113] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2628.202440] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2628.520146] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2628.840221] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2629.138542] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2629.453293] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2629.758294] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2630.090886] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2630.385712] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2630.678252] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2630.968487] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2631.295292] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2631.589544] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2631.894705] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2632.186541] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2632.496292] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2632.784924] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2633.084720] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2633.378875] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2633.691889] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2633.984818] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2634.297985] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2634.602883] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2634.927795] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2635.225492] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2635.528595] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2635.833316] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2636.157795] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2636.467066] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2636.773934] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2637.068333] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2637.383027] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2637.693900] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2638.003494] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2638.297229] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2638.608559] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2638.910269] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2639.217335] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2639.542745] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2639.863826] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2640.179678] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2640.488474] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2640.793516] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2641.124110] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2641.441604] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2641.773460] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2642.092321] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2642.412872] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2642.715570] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2643.017120] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2643.315829] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2643.626205] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2643.932046] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2644.235787] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2644.568394] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2644.929216] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2645.250557] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2645.577199] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2645.897099] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2646.210027] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2646.513200] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2646.822277] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2647.110302] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2647.424359] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2647.740551] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2648.071365] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2648.392387] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2648.740781] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2649.065542] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2649.381446] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2649.683945] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2650.036079] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2650.356125] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2650.668150] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2650.984132] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2651.329106] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2651.653029] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2651.977930] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2652.303960] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2652.636825] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2652.970422] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2653.298211] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2653.622697] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2653.967062] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2654.281202] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2654.607287] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2654.921425] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2655.291539] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2655.615766] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2655.927218] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2656.228483] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2656.545414] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2656.850760] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2657.154304] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2657.466686] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2657.792522] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2658.107041] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2658.408847] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2658.700798] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2659.013800] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2659.303369] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2659.607665] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2659.909317] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2660.219254] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2660.518704] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2660.832251] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2661.130190] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2661.451490] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2661.755410] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2662.057970] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2662.374375] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2662.694177] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2663.004916] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2663.331933] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2663.641691] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2663.995042] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2664.309459] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2664.615245] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2664.930485] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2665.245596] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2665.538524] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2665.834771] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2666.129993] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2666.457470] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2666.760292] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2667.073723] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2667.375709] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2667.687536] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2667.974933] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 2667.977191] Lustre: Skipped 203 previous similar messages [ 2667.981721] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2667.984742] Lustre: Skipped 203 previous similar messages [ 2667.985974] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 2667.987618] Lustre: Skipped 203 previous similar messages [ 2667.988840] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2667.991614] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 2667.993207] Lustre: Skipped 203 previous similar messages [ 2668.276345] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2668.573825] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2668.897871] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2669.211091] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2669.517835] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2669.836207] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2670.185896] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2670.504653] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2670.813222] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2671.132354] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2671.478553] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2671.801541] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2672.136494] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2672.447279] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2672.775952] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2673.085265] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2673.407356] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2673.707070] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2674.027754] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2674.328956] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2674.641785] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2674.949924] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2675.279757] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2675.594278] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2675.916414] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2676.219332] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2676.557642] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2676.888330] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2677.223985] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2677.539400] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2677.872938] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2678.188860] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2678.522351] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2678.854539] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2679.204386] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2679.521990] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2679.844520] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2680.162156] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2680.475135] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2680.770294] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2681.067188] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2681.385344] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2681.717080] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2682.029587] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2682.345289] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2682.662305] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2683.002498] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2683.334631] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2683.649766] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2683.963330] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2684.289598] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2684.600788] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2684.913559] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2685.225785] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2685.547895] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2685.859914] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2686.186039] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2686.494499] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2686.824837] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2687.136132] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2687.447601] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2687.747607] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2688.069294] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2688.382969] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2688.700396] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2689.018613] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2689.358784] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2689.674286] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2689.994390] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2690.309714] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2690.630529] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2690.942563] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2691.253692] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2691.554037] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2691.868921] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2692.172827] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2692.479099] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2692.779131] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2693.111370] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2693.427992] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2693.744363] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2694.060449] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2694.387785] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2694.705844] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2695.010660] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2695.313284] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2695.644847] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2695.970831] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2696.278105] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2696.597601] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2696.924511] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2697.233452] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2697.531938] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2697.837675] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2698.155537] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2698.456947] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2698.768671] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2699.076918] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2699.401479] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2699.713462] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2700.026282] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2700.343895] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2700.662897] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2700.968879] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2701.276989] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2701.580044] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2701.917152] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2702.235888] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2702.537840] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2702.848707] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2703.173809] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2703.497954] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2703.799231] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2704.097661] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2704.415340] LustreError: 162718:0:(osp_precreate.c:651:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 2704.420258] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2704.420443] LustreError: 162718:0:(osp_precreate.c:1355:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 2704.716373] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2705.032903] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2705.335777] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2705.648607] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2705.947345] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2706.243271] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2706.542184] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2706.863685] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2707.176416] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2707.479111] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2707.782270] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2708.111087] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2708.446476] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2708.763413] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2709.071806] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2709.381362] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2709.673583] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2709.978397] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2710.279509] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2710.597379] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2710.901932] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2711.213699] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2711.519247] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2711.841697] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2712.147581] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2712.448337] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2712.752036] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2713.075620] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2713.369202] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2713.677635] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2713.979345] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2714.300752] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2714.603953] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2714.925857] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2715.232704] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2715.546841] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2715.858921] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2716.164539] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2716.465715] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2716.784251] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2717.080642] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2717.399994] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2717.705729] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2718.047101] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2718.360687] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2718.665111] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2718.971735] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2719.292356] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2719.601709] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2719.897833] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2720.197038] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2720.510750] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2720.815286] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2721.120423] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2721.425071] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2721.743889] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2722.059928] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2722.380565] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2722.690554] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2723.013171] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2723.317775] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2723.637463] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2723.950024] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2724.266848] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2724.571225] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2724.881112] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2725.194881] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2725.531210] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2725.842205] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2726.141484] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2726.433231] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2726.761696] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2727.077566] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2727.388354] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2727.690906] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2728.000421] LustreError: 162718:0:(osp_precreate.c:651:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 2728.004268] LustreError: 162718:0:(osp_precreate.c:1355:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 2728.004954] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2728.320844] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2728.632563] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2728.940651] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2729.241412] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2729.538561] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2729.859957] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2730.155506] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2730.477956] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2730.770361] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2731.063434] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2731.369295] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2731.687713] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2732.007208] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2732.317831] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2732.614798] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2732.925742] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2733.224797] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2733.528228] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2733.823916] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2734.144173] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2734.446173] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2734.752338] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2735.052721] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2735.379808] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2735.698629] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2736.026903] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2736.345484] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2736.654938] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2736.956242] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2737.272995] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2737.587304] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2737.901804] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2738.223279] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2738.553458] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2738.864261] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2739.210632] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2739.538299] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2739.841297] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2740.153953] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2740.488055] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2740.813450] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2741.125985] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2741.453542] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2741.807901] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2742.144767] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2742.487628] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2742.798052] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2743.133981] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2743.424038] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2743.720029] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2744.015017] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2744.326757] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2744.647661] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2744.956968] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2745.270970] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2745.622400] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2745.937982] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2746.267117] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2746.580072] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2746.922414] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2747.246491] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2747.566681] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2747.879882] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2748.209799] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2748.516983] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2748.854933] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2749.167302] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2749.508985] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2749.829911] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2750.155714] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2750.495210] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2750.834688] LustreError: 162718:0:(osp_precreate.c:651:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 2750.838717] LustreError: 162718:0:(osp_precreate.c:1355:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 2750.839678] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2751.158870] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2751.461286] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2751.790616] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2752.127628] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2752.443357] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2752.747225] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2753.057245] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2753.379508] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2753.699337] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2754.009703] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2754.313144] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2754.659179] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2754.997278] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2755.330364] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2755.668750] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2756.005104] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2756.307060] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2756.621310] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2756.939315] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2757.265873] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2757.578577] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2757.886778] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2758.199674] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2758.532747] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2758.834185] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2759.138400] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2759.448300] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2759.784841] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2760.105566] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2760.425595] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2760.739812] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2761.062563] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2761.381230] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2761.702978] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2762.009655] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2762.345925] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2762.664612] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2762.984351] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2763.287309] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2763.617301] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2763.940717] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2764.288032] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2764.610041] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2764.962968] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2765.285480] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2765.620990] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2765.946256] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2766.276808] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2766.598305] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2766.925644] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2767.262397] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2767.601642] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2767.914255] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2768.250730] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2768.570242] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2768.905718] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2769.234319] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2769.562461] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2769.868823] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2770.204990] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2770.517729] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2770.829312] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2771.159456] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2771.491255] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2771.806264] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2772.126879] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2772.463135] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2772.809505] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2773.145205] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2773.491833] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2773.840213] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2774.169710] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2774.507172] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2774.853263] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2775.183428] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2775.526641] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2775.855935] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2776.196621] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2776.533934] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2776.860517] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2777.221719] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2777.548954] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2777.863764] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2778.206598] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2778.544898] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2778.887429] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2779.208180] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2779.522452] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2779.863953] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2780.207302] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2780.555383] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2780.894922] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2781.223715] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2781.573569] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2781.928836] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2782.286214] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2782.606215] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2782.909136] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2783.218048] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2783.534795] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2783.853540] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2784.152818] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2784.450195] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2784.762524] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2785.066299] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2785.369579] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2785.679855] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2785.984944] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2786.275883] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2786.565606] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2786.843480] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2787.141037] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2787.424620] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2787.721938] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2788.004925] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2788.310513] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2788.608520] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2788.912366] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2789.234517] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2789.543317] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2789.857466] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2790.155156] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2790.473915] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2790.805453] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2791.112525] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2791.399963] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2791.695326] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2792.006208] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2792.306791] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2792.610657] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2792.918887] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2793.228828] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2793.521991] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2793.828474] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2794.113813] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2794.410900] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2794.682262] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2794.959088] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2795.256037] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2795.554421] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2795.854910] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2796.125757] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 2796.127402] Lustre: Skipped 404 previous similar messages [ 2796.131903] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2796.134748] Lustre: Skipped 404 previous similar messages [ 2796.136083] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 2796.137440] Lustre: Skipped 404 previous similar messages [ 2796.138466] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2796.140991] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 2796.143696] Lustre: Skipped 404 previous similar messages [ 2796.416093] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2796.714085] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2797.054048] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2797.338996] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2797.637539] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2797.961898] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2798.256111] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2798.558090] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2798.834031] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2799.144693] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2799.429050] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2799.750864] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2800.029570] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2800.347588] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2800.656970] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2800.963631] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2801.290995] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2801.637797] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2801.951946] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2802.258818] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2802.559132] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2802.879390] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2803.181299] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2803.484372] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2803.799547] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2804.118503] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2804.426498] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2804.730402] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2805.022270] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2805.332129] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2805.637344] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2805.945213] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2806.237479] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2806.540896] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2806.831242] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2807.125115] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2807.434608] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2807.733403] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2808.025597] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2808.310164] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2808.590597] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2808.905196] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2809.189279] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2809.459634] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2809.755141] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2810.063543] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2810.345817] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2810.636588] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2810.936099] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2811.222982] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2811.497480] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2811.789093] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2812.115277] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2812.433942] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2812.740273] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2813.040281] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2813.337480] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2813.646238] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2813.941915] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2814.258168] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2814.535832] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2814.853312] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2815.146881] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2815.457091] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2815.771483] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2816.089809] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2816.366144] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2816.640368] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2816.917189] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2817.199544] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2817.485368] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2817.772849] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2818.044414] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2818.323637] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2818.600938] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2818.866225] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2819.140346] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2819.412098] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2819.674650] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2819.941718] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2820.218717] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2820.550291] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2820.887502] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2821.197481] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2821.498134] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2821.816480] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2822.113922] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2822.412827] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2822.712699] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2823.035860] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2823.353617] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2823.675558] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2823.997226] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2824.317663] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2824.624663] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2824.990736] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2825.324307] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2825.674805] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2825.995449] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2826.296505] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2826.625243] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2826.981707] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2827.294155] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2827.602421] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2827.903665] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2828.217682] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2828.536544] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2828.845652] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2829.171137] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2829.522235] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2829.843667] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2830.178934] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2830.508398] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2830.847269] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2831.187604] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2831.500735] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2831.830200] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2832.157303] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2832.464253] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2832.782891] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2833.096186] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2833.441794] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2833.782437] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2834.107376] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2834.441478] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2834.784342] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2835.110775] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2835.419336] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2835.738948] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2836.064247] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2836.379168] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2836.693372] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2837.028206] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2837.396988] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2837.733476] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2838.090259] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2838.422462] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2838.748433] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2839.090248] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2839.427369] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2839.780384] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2840.149735] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2840.472556] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2840.810336] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2841.128118] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2841.452670] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2841.783221] LustreError: 162718:0:(osp_precreate.c:651:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 2841.786867] LustreError: 162718:0:(osp_precreate.c:1355:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 2841.788814] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2842.108326] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2842.428109] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2842.747833] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2843.079916] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2843.409756] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2843.750927] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2844.056056] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2844.407914] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2844.761086] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2845.063844] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2845.399865] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2845.787027] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2846.111754] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2846.427829] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2846.762092] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2847.093346] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2847.404325] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2847.722121] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2848.043372] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2848.369831] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2848.682062] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2849.011909] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2849.325481] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2849.643113] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2849.972115] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2850.285457] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2850.622875] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2850.983678] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2851.304261] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2851.631478] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2851.951436] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2852.279905] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2852.586398] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2852.892191] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2913.018836] LustreError: 164296:0:(ofd_dev.c:1780:ofd_create_hdl()) lustre-OST0000: unable to precreate: rc = -28 [ 2913.021575] LustreError: 162718:0:(osp_precreate.c:651:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -28 [ 2976.227431] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2976.228582] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2976.231382] Lustre: Skipped 1 previous similar message [ 2981.346721] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2981.349793] Lustre: Skipped 1 previous similar message [ 2981.501089] LustreError: 216442:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 2981.563060] Lustre: server umount lustre-OST0000 complete [ 2986.465897] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2986.466866] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2986.470464] Lustre: Skipped 1 previous similar message [ 2991.585168] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2991.586893] Lustre: Skipped 1 previous similar message [ 2996.705260] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2996.708063] Lustre: Skipped 1 previous similar message [ 2997.216080] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2997.254939] LustreError: 216643:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup [ 2997.256489] LustreError: 216643:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 2997.340570] Lustre: server umount lustre-MDT0000 complete [ 2998.554205] LustreError: 162680:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1767037491 with bad export cookie 16927598187177741725 [ 2998.558137] LustreError: MGC192.168.204.129@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2998.617402] LustreError: 216846:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [ 2998.619233] LustreError: 216846:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 2998.670380] Lustre: server umount lustre-MDT0001 complete [ 3002.218988] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing unload_modules_local [ 3003.253725] Key type lgssc unregistered [ 3003.388349] LNet: 217377:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3003.392196] LNetError: 217377:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3003.400319] LNet: Removed LNI 192.168.204.129@tcp [ 3003.684964] Key type .llcrypt unregistered [ 3003.685953] Key type ._llcrypt unregistered [ 3012.917070] Key type ._llcrypt registered [ 3012.918014] Key type .llcrypt registered [ 3012.954393] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing load_modules_local [ 3013.363628] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3013.379201] alg: No test for adler32 (adler32-zlib) [ 3014.231469] Lustre: Lustre: Build Version: 2.17.0_RC3_1_ga0d6154 [ 3014.311275] LNet: Added LNI 192.168.204.129@tcp [8/256/0/180] [ 3015.888196] Key type lgssc registered [ 3016.243185] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3018.733789] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3018.741359] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3019.894963] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3021.226182] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 3024.042346] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3024.180193] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3025.571448] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 3027.300699] Lustre: DEBUG MARKER: oleg429-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 3028.477338] Lustre: DEBUG MARKER: oleg429-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 3030.761186] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3030.872779] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3032.782205] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 3034.556795] Lustre: DEBUG MARKER: oleg429-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3044.201844] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49666 to 0x280000401:49697) [ 3047.827785] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 3048.450484] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3048.457191] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 3048.461697] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3048.464809] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 3048.464821] LustreError: 218767:0:(osp_precreate.c:1263:osp_precreate_thread()) cfs_fail_timeout id 2107 sleeping for 20000ms [ 3049.442912] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3049.443818] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3049.446286] Lustre: Skipped 1 previous similar message [ 3049.450966] Lustre: Skipped 1 previous similar message [ 3050.246630] Lustre: lustre-OST0000: Not available for connect from 192.168.204.29@tcp (stopping) [ 3051.119636] LustreError: 221167:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 3051.159432] Lustre: server umount lustre-OST0000 complete [ 3068.544098] LustreError: 218767:0:(osp_precreate.c:1263:osp_precreate_thread()) cfs_fail_timeout id 2107 awake [ 3079.183358] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3079.293945] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3081.112548] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 3082.898803] Lustre: DEBUG MARKER: oleg429-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3089.573774] LustreError: lustre-OST0000-osc-MDT0001: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3089.577221] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3089.577300] Lustre: lustre-OST0000-osc-MDT0001: Connection restored to 0@lo (at 0@lo) [ 3089.582123] Lustre: Skipped 1 previous similar message [ 3092.635468] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid in FULL state after 9 sec [ 3093.869893] Lustre: DEBUG MARKER: oleg429-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff8a61c783b000.ost_server_uuid 50 [ 3094.424743] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff8a61c783b000.ost_server_uuid in FULL state after 0 sec [ 3100.130122] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3100.132362] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3100.136979] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3100.138675] Lustre: Skipped 1 previous similar message [ 3101.550430] LustreError: 222485:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 3101.552109] LustreError: 222485:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 3101.583452] Lustre: server umount lustre-OST0000 complete [ 3105.249904] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3105.250670] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3105.254624] Lustre: Skipped 2 previous similar messages [ 3105.258512] Lustre: Skipped 1 previous similar message [ 3108.927276] LustreError: 222687:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup [ 3108.930148] LustreError: 222687:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 3108.960693] Lustre: server umount lustre-MDT0000 complete [ 3110.250420] LustreError: 218730:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1767037603 with bad export cookie 14128281584433957033 [ 3110.254320] LustreError: MGC192.168.204.129@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3110.255702] LustreError: 218730:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 3110.394169] Lustre: server umount lustre-MDT0001 complete [ 3114.181107] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing unload_modules_local [ 3115.236108] Key type lgssc unregistered [ 3115.369340] LNet: 223418:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3115.371697] LNetError: 223418:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3115.380336] LNet: Removed LNI 192.168.204.129@tcp [ 3115.689580] Key type .llcrypt unregistered [ 3115.691132] Key type ._llcrypt unregistered [ 3127.243610] Key type ._llcrypt registered [ 3127.244678] Key type .llcrypt registered [ 3127.282095] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing load_modules_local [ 3127.620370] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3127.668363] alg: No test for adler32 (adler32-zlib) [ 3128.528914] Lustre: Lustre: Build Version: 2.17.0_RC3_1_ga0d6154 [ 3128.614821] LNet: Added LNI 192.168.204.129@tcp [8/256/0/180] [ 3130.200148] Key type lgssc registered [ 3130.623786] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3133.720812] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3133.726837] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3134.880654] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3136.273591] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 3139.135559] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3139.277221] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3140.621651] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 3141.670623] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 3143.923421] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3144.042472] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3145.066725] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49699 to 0x280000401:49729) [ 3146.311593] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 3149.126828] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3149.151139] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3149.218610] Lustre: lustre-OST0001: new disk, initializing [ 3149.220512] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 3149.247178] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 3151.173967] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 3156.568106] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 3156.996477] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 3156.999866] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 3157.036810] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 3161.200761] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 3172.323231] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3172.325437] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3172.328564] Lustre: Skipped 2 previous similar messages [ 3172.330564] Lustre: Skipped 1 previous similar message [ 3177.442319] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3177.444257] Lustre: Skipped 2 previous similar messages [ 3177.603520] LustreError: 228467:0:(obd_class.h:479:obd_check_dev()) Device 28 not setup [ 3177.639705] Lustre: server umount lustre-MDT0000 complete [ 3178.881548] LustreError: 227385:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1767037671 with bad export cookie 5324394873331561878 [ 3178.888084] LustreError: MGC192.168.204.129@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3178.955245] LustreError: 228668:0:(obd_class.h:479:obd_check_dev()) Device 27 not setup [ 3178.957204] LustreError: 228668:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 3179.021942] Lustre: server umount lustre-MDT0001 complete [ 3190.772078] LustreError: 228869:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 3190.774038] LustreError: 228869:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 3190.791596] Lustre: server umount lustre-OST0000 complete [ 3201.268378] LustreError: 229071:0:(obd_class.h:479:obd_check_dev()) Device 2 not setup [ 3201.271433] LustreError: 229071:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 3201.320299] Lustre: server umount lustre-OST0001 complete [ 3202.427933] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro [ 3202.454535] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3202.817257] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro [ 3202.847257] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3203.190096] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: ,errors=remount-ro [ 3203.215054] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3203.566435] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: ,errors=remount-ro [ 3203.594473] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3208.112348] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing load_modules_local [ 3211.433975] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3211.584577] LustreError: 230478:0:(ldlm_lib.c:1179:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3211.593224] LustreError: 230478:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 3211.610416] Lustre: mylustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3213.005714] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 3215.824283] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3217.189699] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 3218.131072] Lustre: 231588:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 3220.271279] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3220.389857] Lustre: mylustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3220.391966] Lustre: Skipped 1 previous similar message [ 3221.413901] LustreError: 231944:0:(ldlm_lib.c:1179:target_handle_connect()) mylustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3221.417198] Lustre: mylustre-OST0000: new connection from mylustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49732 to 0x280000401:49761) [ 3222.227785] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 3225.016358] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3226.887378] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 3229.938976] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 3231.091678] Lustre: 233432:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 3241.444592] Lustre: mylustre-MDT0000-lwp-MDT0001: Connection to mylustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3241.445519] LustreError: mylustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3241.446323] Lustre: mylustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3241.448816] Lustre: Skipped 2 previous similar messages [ 3246.335966] LustreError: 233741:0:(obd_class.h:479:obd_check_dev()) Device 10 not setup [ 3246.338185] LustreError: 233741:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 3246.373614] Lustre: server umount mylustre-MDT0000 complete [ 3246.565259] LustreError: 230473:0:(ldlm_lib.c:1179:target_handle_connect()) mylustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3246.570379] LustreError: 230473:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 3247.600400] LustreError: 230453:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1767037740 with bad export cookie 5324394873331564650 [ 3247.603782] LustreError: 230453:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 3247.606653] LustreError: MGC192.168.204.129@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3247.740413] Lustre: server umount mylustre-MDT0001 complete [ 3258.868146] LustreError: 234153:0:(obd_class.h:479:obd_check_dev()) Device 23 not setup [ 3258.869887] LustreError: 234153:0:(obd_class.h:479:obd_check_dev()) Skipped 15 previous similar messages [ 3258.885401] Lustre: server umount mylustre-OST0000 complete [ 3270.362104] Lustre: server umount mylustre-OST0001 complete [ 3271.332451] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro [ 3271.359279] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3271.701920] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro [ 3271.731242] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3272.073057] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: ,errors=remount-ro [ 3272.095855] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3272.437212] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: ,errors=remount-ro [ 3272.463160] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3276.781957] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing load_modules_local [ 3280.085041] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3280.234098] LustreError: 235762:0:(ldlm_lib.c:1179:target_handle_connect()) tfs-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3280.258540] Lustre: tfs-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3280.261723] Lustre: Skipped 1 previous similar message [ 3281.527820] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 3284.320535] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3285.704050] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 3286.634940] Lustre: 236872:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 3288.763870] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3290.664662] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 3290.917679] LustreError: 237226:0:(ldlm_lib.c:1179:target_handle_connect()) tfs-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3290.924565] Lustre: tfs-OST0000: new connection from tfs-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49732 to 0x280000401:49793) [ 3293.480692] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3294.636904] Lustre: tfs-OST0001: new connection from tfs-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:35 to 0x2c0000401:65) [ 3295.479662] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 3298.586137] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 3299.782194] Lustre: 238717:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 3310.049611] LustreError: tfs-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3310.052690] Lustre: tfs-MDT0000-osp-MDT0001: Connection to tfs-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3310.054448] Lustre: tfs-MDT0000: Not available for connect from 0@lo (stopping) [ 3310.057260] Lustre: Skipped 4 previous similar messages [ 3310.058788] Lustre: Skipped 5 previous similar messages [ 3312.703103] LustreError: 238965:0:(obd_class.h:479:obd_check_dev()) Device 10 not setup [ 3312.706123] LustreError: 238965:0:(obd_class.h:479:obd_check_dev()) Skipped 9 previous similar messages [ 3312.733088] Lustre: server umount tfs-MDT0000 complete [ 3313.922066] LustreError: 235736:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1767037806 with bad export cookie 5324394873331566323 [ 3313.925316] LustreError: MGC192.168.204.129@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3313.925372] LustreError: 235736:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 3314.044589] Lustre: server umount tfs-MDT0001 complete [ 3325.507214] Lustre: server umount tfs-OST0000 complete [ 3336.986316] Lustre: server umount tfs-OST0001 complete [ 3338.017930] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro [ 3338.042988] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3338.392298] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro [ 3338.422913] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3338.777889] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: ,errors=remount-ro [ 3338.799819] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3339.147979] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: ,errors=remount-ro [ 3339.179816] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3343.633040] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing load_modules_local [ 3347.005333] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3347.160420] LustreError: 240977:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3347.185373] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3347.188040] Lustre: Skipped 3 previous similar messages [ 3348.565754] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 3351.372134] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3352.802393] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 3353.750142] Lustre: 242090:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 3355.950801] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3357.984484] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 3358.125037] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49732 to 0x280000401:49825) [ 3360.920303] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3362.030861] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:35 to 0x2c0000401:97) [ 3362.856369] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 3365.897252] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 3367.049819] Lustre: 243934:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 3374.304267] Lustre: DEBUG MARKER: == conf-sanity test 104a: Make sure user defined options are reflected in mount ========================================================== 14:51:06 (1767037866) [ 3387.361028] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3387.363630] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3387.368516] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3387.372081] Lustre: Skipped 1 previous similar message [ 3392.320182] LustreError: 244304:0:(obd_class.h:479:obd_check_dev()) Device 10 not setup [ 3392.322208] LustreError: 244304:0:(obd_class.h:479:obd_check_dev()) Skipped 25 previous similar messages [ 3392.351644] Lustre: server umount lustre-MDT0000 complete [ 3392.996676] LustreError: 240971:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3393.000845] LustreError: 240971:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 4 previous similar messages [ 3393.536422] LustreError: 240951:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1767037886 with bad export cookie 5324394873331567989 [ 3393.540424] LustreError: MGC192.168.204.129@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3393.541696] LustreError: 240951:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 3398.113874] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3398.114697] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 3398.119876] Lustre: Skipped 4 previous similar messages [ 3398.121451] Lustre: Skipped 3 previous similar messages [ 3399.730717] Lustre: server umount lustre-MDT0001 complete [ 3407.114997] Lustre: server umount lustre-OST0000 complete [ 3414.420316] Lustre: server umount lustre-OST0001 complete [ 3416.548125] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_hostid [ 3419.668138] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing load_modules_local [ 3422.925509] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3425.121288] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3427.254949] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3429.415200] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3431.754169] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3431.778792] LDISKFS-fs (dm-0): Mount option "noacl" will be removed by 3.5 [ 3431.778792] Contact linux-ldiskfs@vger.kernel.org if you think we should keep it. [ 3431.778792] [ 3431.788348] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: acl,user_xattr,errors=remount-ro,noacl,no_mbcache,nodelalloc [ 3431.887649] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3431.900569] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3431.937390] Lustre: lustre-MDT0000: new disk, initializing [ 3431.963327] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3431.965443] Lustre: Skipped 3 previous similar messages [ 3431.971722] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3433.383117] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 3437.689642] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3437.718666] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3437.752553] Lustre: 247088:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 3437.781667] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 3437.783816] Lustre: Skipped 1 previous similar message [ 3437.822610] Lustre: lustre-MDT0001: new disk, initializing [ 3437.877572] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 3437.880805] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3439.256904] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 3443.569397] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3443.604061] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3443.719504] Lustre: lustre-OST0000: new disk, initializing [ 3443.721690] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3444.803993] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 3444.807680] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 3444.852408] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3445.780932] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 3450.085567] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3450.109912] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3450.174265] Lustre: lustre-OST0001: new disk, initializing [ 3450.176620] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 3451.524915] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 3451.527983] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 3451.566164] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 3452.222107] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 3456.652310] Lustre: DEBUG MARKER: == conf-sanity test 104b: Mount uses last flock argument ========================================================== 14:52:28 (1767037948) [ 3460.397569] Lustre: DEBUG MARKER: == conf-sanity test 105: check file creation for ro and rw bind mnt pt ========================================================== 14:52:32 (1767037952) [ 3472.359177] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3472.361871] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3472.362734] Lustre: Skipped 1 previous similar message [ 3472.365661] Lustre: Skipped 2 previous similar messages [ 3486.176148] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 3486.195107] LustreError: 250513:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 3486.198148] LustreError: 250513:0:(obd_class.h:479:obd_check_dev()) Skipped 25 previous similar messages [ 3486.216771] Lustre: server umount lustre-OST0000 complete [ 3487.716318] LustreError: 250514:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3487.720955] LustreError: 250514:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 3493.731386] Lustre: server umount lustre-MDT0000 complete [ 3495.056197] LustreError: 250414:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1767037987 with bad export cookie 5324394873331569543 [ 3495.059863] LustreError: 250414:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 3495.060084] LustreError: MGC192.168.204.129@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3497.953775] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3497.959529] Lustre: Skipped 3 previous similar messages [ 3501.253337] Lustre: server umount lustre-MDT0001 complete [ 3504.951660] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing unload_modules_local [ 3523.040139] Lustre: lustre-OST0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 3523.097721] Lustre: server umount lustre-OST0001 complete [ 3525.113809] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_hostid [ 3531.565605] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing load_modules_local [ 3534.937570] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3537.106136] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3539.206131] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3541.343146] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3543.692022] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3543.714584] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3543.802113] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3543.813232] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3543.853100] Lustre: lustre-MDT0000: new disk, initializing [ 3543.883216] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3545.228455] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 3549.523364] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3549.552030] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3549.589883] Lustre: 253974:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 3549.698132] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3551.067436] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 3554.218660] Lustre: DEBUG MARKER: oleg429-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 3555.495179] Lustre: DEBUG MARKER: oleg429-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 3557.789271] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3557.814448] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3559.099024] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 3559.138193] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3559.912393] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 3563.010581] Lustre: DEBUG MARKER: oleg429-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3569.634957] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3569.636378] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3569.636917] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3569.636923] Lustre: Skipped 12 previous similar messages [ 3569.638145] LustreError: Skipped 1 previous similar message [ 3569.646355] Lustre: Skipped 1 previous similar message [ 3571.405965] Lustre: server umount lustre-OST0000 complete [ 3578.790643] Lustre: server umount lustre-MDT0000 complete [ 3579.873641] LustreError: 253983:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3579.878334] LustreError: 253983:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 4 previous similar messages [ 3580.021619] LustreError: 253966:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1767038072 with bad export cookie 5324394873331571013 [ 3580.026400] LustreError: MGC192.168.204.129@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3580.154328] Lustre: server umount lustre-MDT0001 complete [ 3583.662225] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing unload_modules_local [ 3584.764227] Key type lgssc unregistered [ 3584.903468] LNet: 257351:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3584.905571] LNetError: 257351:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3584.912566] LNet: Removed LNI 192.168.204.129@tcp [ 3585.243138] Key type .llcrypt unregistered [ 3585.244850] Key type ._llcrypt unregistered [ 3595.135749] Key type ._llcrypt registered [ 3595.137339] Key type .llcrypt registered [ 3595.176037] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing load_modules_local [ 3595.476636] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3595.494048] alg: No test for adler32 (adler32-zlib) [ 3596.342659] Lustre: Lustre: Build Version: 2.17.0_RC3_1_ga0d6154 [ 3596.432831] LNet: Added LNI 192.168.204.129@tcp [8/256/0/180] [ 3598.008088] Key type lgssc registered [ 3598.347223] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3600.863436] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3600.869491] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3602.024285] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3603.271411] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 3606.048409] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3606.165668] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3607.484917] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 3609.166733] Lustre: DEBUG MARKER: oleg429-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 3610.368552] Lustre: DEBUG MARKER: oleg429-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 3612.701607] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3612.824449] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3614.596924] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 3616.303323] Lustre: DEBUG MARKER: oleg429-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3616.638071] Lustre: Setting parameter lustre-OST0000.ost.unknown_param=50 in log lustre-OST0000 [ 3622.371825] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3622.372648] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3622.378106] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3623.341081] LustreError: 261065:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 3623.380325] Lustre: server umount lustre-OST0000 complete [ 3626.976822] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3626.978787] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3626.985302] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3630.780091] LustreError: 261270:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup [ 3630.781661] LustreError: 261270:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 3630.806930] Lustre: server umount lustre-MDT0000 complete [ 3632.063213] LustreError: 258750:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1767038124 with bad export cookie 11875515225641958910 [ 3632.066975] LustreError: MGC192.168.204.129@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3632.067070] LustreError: 258750:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 3632.135062] LustreError: 261473:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [ 3632.136759] LustreError: 261473:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 3632.190073] Lustre: server umount lustre-MDT0001 complete [ 3635.847587] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing unload_modules_local [ 3636.859648] Key type lgssc unregistered [ 3636.977513] LNet: 262004:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3636.979533] LNetError: 262004:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3636.990414] LNet: Removed LNI 192.168.204.129@tcp [ 3637.316508] Key type .llcrypt unregistered [ 3637.317448] Key type ._llcrypt unregistered [ 3644.230579] Key type ._llcrypt registered [ 3644.231531] Key type .llcrypt registered [ 3644.270111] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing load_modules_local [ 3644.593290] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3644.617098] alg: No test for adler32 (adler32-zlib) [ 3645.476817] Lustre: Lustre: Build Version: 2.17.0_RC3_1_ga0d6154 [ 3645.574044] LNet: Added LNI 192.168.204.129@tcp [8/256/0/180] [ 3647.160149] Key type lgssc registered [ 3647.525721] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3650.495208] Lustre: lustre-OST0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3650.500783] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3665.952322] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 3671.072178] LustreError: 263154:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.204.129@tcp: failed processing log, type 4: rc = -110 [ 3697.696177] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3699.528567] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 3701.264725] Lustre: DEBUG MARKER: oleg429-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3702.001432] LustreError: 263818:0:(obd_class.h:479:obd_check_dev()) Device 1 not setup [ 3702.041846] Lustre: server umount lustre-OST0000 complete [ 3707.696168] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing unload_modules_local [ 3708.749294] Key type lgssc unregistered [ 3708.867377] LNet: 264654:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3708.869837] LNetError: 264654:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3708.880271] LNet: Removed LNI 192.168.204.129@tcp [ 3709.156274] Key type .llcrypt unregistered [ 3709.157248] Key type ._llcrypt unregistered [ 3722.670894] Key type ._llcrypt registered [ 3722.671869] Key type .llcrypt registered [ 3722.711917] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing load_modules_local [ 3723.221759] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3723.240596] alg: No test for adler32 (adler32-zlib) [ 3724.088101] Lustre: Lustre: Build Version: 2.17.0_RC3_1_ga0d6154 [ 3724.168054] LNet: Added LNI 192.168.204.129@tcp [8/256/0/180] [ 3725.744195] Key type lgssc registered [ 3726.104486] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3730.303743] loop0: detected capacity change from 0 to 204800000 [ 3730.362634] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 3730.366948] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 3730.490522] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 3730.512458] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3730.853522] loop0: detected capacity change from 0 to 204800000 [ 3730.908291] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 3730.911441] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 3731.035641] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 3731.055654] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3731.419313] loop0: detected capacity change from 0 to 204800000 [ 3731.488180] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 3731.491833] blk_update_request: operation not supported error, dev loop0, sector 8224 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 3731.533499] blk_update_request: operation not supported error, dev loop0, sector 58264 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 3731.577375] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3731.964537] loop0: detected capacity change from 0 to 204800000 [ 3732.024845] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 3732.108505] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3732.439588] loop0: detected capacity change from 0 to 204800000 [ 3732.448325] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 3733.756605] loop0: detected capacity change from 0 to 204800000 [ 3733.767335] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 3735.045317] loop0: detected capacity change from 0 to 204800000 [ 3735.058473] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 3736.362301] loop0: detected capacity change from 0 to 204800000 [ 3736.373632] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 3737.727131] loop0: detected capacity change from 0 to 204800000 [ 3737.766336] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3737.795605] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3737.801487] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3737.825150] Lustre: 268032:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 3737.827991] Lustre: 268032:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index 'nodemap' with [0x200000003:0x3:0x0]: rc = 0 [ 3737.831193] Lustre: 268032:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000003:0xb:0x0]: rc = 0 [ 3737.834306] Lustre: 268032:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x2010000-MDT0000' with [0x200000005:0x3:0x0]: rc = 0 [ 3737.837694] Lustre: 268032:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x1010000-MDT0000' with [0x200000005:0x2:0x0]: rc = 0 [ 3737.840721] Lustre: 268032:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000003:0xa:0x0]: rc = 0 [ 3737.844241] Lustre: 268032:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x10000-MDT0000' with [0x200000005:0x1:0x0]: rc = 0 [ 3737.847343] Lustre: 268032:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000003:0xc:0x0]: rc = 0 [ 3737.851546] Lustre: 268032:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x20000' with [0x200000006:0x20000:0x0]: rc = 0 [ 3737.854488] Lustre: 268032:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x2020000' with [0x200000006:0x2020000:0x0]: rc = 0 [ 3737.858274] Lustre: 268032:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x1020000' with [0x200000006:0x1020000:0x0]: rc = 0 [ 3737.861514] Lustre: 268032:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000006:0x1010000:0x0]: rc = 0 [ 3737.865485] Lustre: 268032:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000006:0x10000:0x0]: rc = 0 [ 3737.869235] Lustre: 268032:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000006:0x2010000:0x0]: rc = 0 [ 3740.334208] LustreError: 268299:0:(obd_class.h:479:obd_check_dev()) Device 2 not setup [ 3740.377857] Lustre: server umount MGS complete [ 3740.655160] loop0: detected capacity change from 0 to 204800000 [ 3740.686711] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3740.843583] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3740.859855] Lustre: lustre-MDT0000: Aborting client recovery [ 3741.158229] loop1: detected capacity change from 0 to 204800000 [ 3741.218696] LDISKFS-fs (loop1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3741.245159] LDISKFS-fs (loop1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3741.267670] Lustre: 268468:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 3741.270814] Lustre: 268468:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0001: restore index '0x2010000' with [0x200000003:0x6:0x0]: rc = 0 [ 3741.273598] Lustre: 268468:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0001: restore index '0x1010000' with [0x200000003:0x5:0x0]: rc = 0 [ 3741.276331] Lustre: 268468:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0001: restore index '0x1010000-MDT0001' with [0x200000005:0x2:0x0]: rc = 0 [ 3741.279409] Lustre: 268468:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0001: restore index '0x2010000-MDT0001' with [0x200000005:0x3:0x0]: rc = 0 [ 3741.284336] Lustre: 268468:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0001: restore index '0x10000-MDT0001' with [0x200000005:0x1:0x0]: rc = 0 [ 3741.288452] Lustre: 268468:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0001: restore index '0x10000' with [0x200000003:0x4:0x0]: rc = 0 [ 3741.294368] Lustre: Found index 1 for lustre-MDT0001, updating log [ 3741.361219] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3741.390890] Lustre: lustre-MDT0001: Aborting client recovery [ 3741.691686] loop2: detected capacity change from 0 to 204800000 [ 3741.737415] LDISKFS-fs (loop2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3741.760550] LDISKFS-fs (loop2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3741.785275] Lustre: 268554:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 3741.788330] Lustre: 268554:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0000: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 3741.791379] Lustre: 268554:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0000: restore index '0x1020000-OST0000' with [0x200000005:0x2:0x0]: rc = 0 [ 3741.794389] Lustre: 268554:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0000: restore index '0x20000-OST0000' with [0x200000005:0x1:0x0]: rc = 0 [ 3741.797176] Lustre: 268554:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0000: restore index '0x2020000-OST0000' with [0x200000005:0x3:0x0]: rc = 0 [ 3741.800314] Lustre: 268554:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0000: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 3741.802988] Lustre: 268554:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0000: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 3741.849063] Lustre: Found index 0 for lustre-OST0000, updating log [ 3741.884943] Lustre: lustre-OST0000: Aborting recovery [ 3742.210697] loop3: detected capacity change from 0 to 204800000 [ 3742.257474] LDISKFS-fs (loop3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3742.321585] LDISKFS-fs (loop3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3742.336222] Lustre: 268642:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 3742.339260] Lustre: 268642:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0001: restore index '0x20000-OST0001' with [0x200000005:0x1:0x0]: rc = 0 [ 3742.342134] Lustre: 268642:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0001: restore index '0x1020000-OST0001' with [0x200000005:0x2:0x0]: rc = 0 [ 3742.344990] Lustre: 268642:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0001: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 3742.347605] Lustre: 268642:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0001: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 3742.350524] Lustre: 268642:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0001: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 3742.353437] Lustre: 268642:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0001: restore index '0x2020000-OST0001' with [0x200000005:0x3:0x0]: rc = 0 [ 3742.387405] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 3742.389904] Lustre: Skipped 1 previous similar message [ 3742.397192] Lustre: lustre-OST0001: Aborting recovery [ 3747.817729] LustreError: 268411:0:(lod_dev.c:510:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 7, retries 0, failed: rc = -108 [ 3750.882886] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 3750.883367] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 3750.893055] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000bd0 [ 3750.896710] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000bd0 [ 3751.907879] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:5 to 0x280000400:33) [ 3751.907962] Lustre: lustre-OST0001: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:5 to 0x2c0000400:33) [ 3753.198548] LustreError: 268379:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0000: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0000 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0000/identity_upcall, time 223us: rc = -2 [ 3767.266100] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3767.266459] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3767.267294] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3767.270924] Lustre: Skipped 1 previous similar message [ 3770.560407] LustreError: 268995:0:(obd_class.h:479:obd_check_dev()) Device 10 not setup [ 3770.564024] LustreError: 268995:0:(obd_class.h:479:obd_check_dev()) Skipped 1 previous similar message [ 3770.602759] Lustre: server umount lustre-MDT0000 complete [ 3770.917063] LustreError: 268478:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1767038263 with bad export cookie 17581409088615879101 [ 3770.919425] LustreError: MGC192.168.204.129@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3770.922389] LustreError: 268478:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 3772.384666] LustreError: 268890:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3772.384791] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3772.385200] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 3772.385205] Lustre: Skipped 3 previous similar messages [ 3772.389363] LustreError: 268890:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 3772.392373] Lustre: Skipped 3 previous similar messages [ 3775.392434] LustreError: 268890:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3775.392578] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 3775.396889] LustreError: 268890:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 3775.398594] Lustre: Skipped 2 previous similar messages [ 3777.151295] LustreError: 269047:0:(obd_class.h:479:obd_check_dev()) Device 18 not setup [ 3777.154354] LustreError: 269047:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 3777.215073] Lustre: server umount lustre-MDT0001 complete [ 3783.597086] LustreError: 269098:0:(obd_class.h:479:obd_check_dev()) Device 23 not setup [ 3783.600656] LustreError: 269098:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 3783.629885] Lustre: server umount lustre-OST0000 complete [ 3789.997657] LustreError: 269149:0:(obd_class.h:479:obd_check_dev()) Device 2 not setup [ 3790.000423] LustreError: 269149:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 3790.045785] Lustre: server umount lustre-OST0001 complete [ 3792.694554] Lustre: DEBUG MARKER: == conf-sanity test 109a: test lctl clear_conf fsname ==== 14:58:04 (1767038284) [ 3801.499515] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_hostid [ 3804.498525] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing load_modules_local [ 3807.791302] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3809.822452] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3811.774848] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3813.850537] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3816.038716] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3816.063079] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3816.144634] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3816.154529] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3816.191141] Lustre: lustre-MDT0000: new disk, initializing [ 3816.214289] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3816.221563] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3817.502840] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 3821.617168] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3821.638819] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3821.669323] Lustre: 272693:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 3821.691433] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 3821.693188] Lustre: Skipped 1 previous similar message [ 3821.729687] Lustre: lustre-MDT0001: new disk, initializing [ 3821.761601] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3821.773033] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 3821.776103] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3823.034902] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 3826.154758] Lustre: DEBUG MARKER: oleg429-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 3827.336982] Lustre: DEBUG MARKER: oleg429-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 3829.534581] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3829.559113] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3829.657408] Lustre: lustre-OST0000: new disk, initializing [ 3829.659705] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3831.491958] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 3834.525890] Lustre: DEBUG MARKER: oleg429-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3836.924047] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 3836.927205] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 3836.959747] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3841.952523] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [ 3844.415816] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [ 3852.822413] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [ 3860.362925] Lustre: Modifying parameter lustre-client.llite.max_read_ahead_mb=64 in log lustre-client [ 3893.217800] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3893.220431] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3893.221618] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3893.225888] Lustre: Skipped 1 previous similar message [ 3894.259538] LustreError: 276255:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 3894.261387] LustreError: 276255:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 3894.296936] Lustre: server umount lustre-OST0000 complete [ 3898.337863] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3898.338619] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3898.341188] Lustre: Skipped 1 previous similar message [ 3898.344190] Lustre: Skipped 2 previous similar messages [ 3901.660409] Lustre: server umount lustre-MDT0000 complete [ 3902.869381] LustreError: 276150:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1767038395 with bad export cookie 17581409088615881810 [ 3902.872317] LustreError: 276150:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 3902.873226] LustreError: MGC192.168.204.129@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3902.991288] Lustre: server umount lustre-MDT0001 complete [ 3905.771533] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3907.034963] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 3908.627987] Lustre: server umount MGS complete [ 3912.387918] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3912.533511] LustreError: 278222:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3912.539482] LustreError: 278234:0:(qmt_pool.c:1523:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [ 3912.559201] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3912.561152] Lustre: Skipped 1 previous similar message [ 3913.778560] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 3916.588633] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3917.973151] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 3919.687339] Lustre: DEBUG MARKER: oleg429-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 3920.822664] Lustre: DEBUG MARKER: oleg429-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 3923.001783] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3924.942384] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 3926.757247] Lustre: DEBUG MARKER: oleg429-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3961.314738] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3961.319610] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3961.321777] Lustre: Skipped 1 previous similar message [ 3965.295087] LustreError: 280902:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 3965.296737] LustreError: 280902:0:(obd_class.h:479:obd_check_dev()) Skipped 19 previous similar messages [ 3965.329331] Lustre: server umount lustre-OST0000 complete [ 3967.968739] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3967.970965] LustreError: Skipped 1 previous similar message [ 3971.553891] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3971.557202] Lustre: Skipped 2 previous similar messages [ 3972.694161] Lustre: server umount lustre-MDT0000 complete [ 3973.892186] LustreError: 280816:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1767038466 with bad export cookie 17581409088615883609 [ 3973.895687] LustreError: 280816:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 3973.895727] LustreError: MGC192.168.204.129@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3974.013416] Lustre: server umount lustre-MDT0001 complete [ 3977.460883] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing unload_modules_local [ 3978.425484] Key type lgssc unregistered [ 3978.557339] LNet: 281846:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3978.559550] LNetError: 281846:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3978.567297] LNet: Removed LNI 192.168.204.129@tcp [ 3978.868568] Key type .llcrypt unregistered [ 3978.869499] Key type ._llcrypt unregistered [ 3990.712445] Key type ._llcrypt registered [ 3990.713951] Key type .llcrypt registered [ 3990.752637] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_hostid [ 3996.649635] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing load_modules_local [ 3997.026054] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3997.056694] alg: No test for adler32 (adler32-zlib) [ 3997.910459] Lustre: Lustre: Build Version: 2.17.0_RC3_1_ga0d6154 [ 3997.992290] LNet: Added LNI 192.168.204.129@tcp [8/256/0/180] [ 3999.576143] Key type lgssc registered [ 3999.922263] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4002.762787] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4004.930405] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4006.887655] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4009.265772] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4011.557804] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4011.577127] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4011.581583] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4012.668104] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 4012.680823] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 4012.722387] Lustre: lustre-MDT0000: new disk, initializing [ 4012.748486] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4012.756523] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 4014.060147] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 4018.169904] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4018.192300] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4018.224800] Lustre: 285751:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 4018.251171] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 4018.252777] Lustre: Skipped 1 previous similar message [ 4018.287123] Lustre: lustre-MDT0001: new disk, initializing [ 4018.319479] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4018.330785] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 4018.334661] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 4019.595551] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 4022.622830] Lustre: DEBUG MARKER: oleg429-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 4023.777064] Lustre: DEBUG MARKER: oleg429-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 4026.053867] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4026.078131] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4026.181673] Lustre: lustre-OST0000: new disk, initializing [ 4026.183346] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 4026.209803] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4027.996416] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 4031.012823] Lustre: DEBUG MARKER: oleg429-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4031.480596] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 4031.484247] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 4031.514859] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 4039.621076] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [ 4048.491725] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [ 4055.548872] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [ 4060.020345] Lustre: Modifying parameter lustre-client.llite.max_read_ahead_mb=64 in log lustre-client [ 4098.017894] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4098.020062] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4098.020689] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4098.023043] Lustre: Skipped 1 previous similar message [ 4103.138953] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4103.141618] Lustre: Skipped 2 previous similar messages [ 4108.259633] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4108.262469] Lustre: Skipped 1 previous similar message [ 4112.352108] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 4112.369062] LustreError: 289507:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 4112.409234] Lustre: server umount lustre-OST0000 complete [ 4115.936917] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4115.941116] LustreError: Skipped 1 previous similar message [ 4115.943587] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4115.950484] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4118.497836] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4119.744745] LustreError: 289710:0:(obd_class.h:479:obd_check_dev()) Device 22 not setup [ 4119.747939] LustreError: 289710:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 4119.772106] Lustre: server umount lustre-MDT0000 complete [ 4121.066234] LustreError: 285742:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1767038613 with bad export cookie 13945889275545175055 [ 4121.070523] LustreError: MGC192.168.204.129@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4121.140994] LustreError: 289911:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 4121.142714] LustreError: 289911:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 4121.193449] Lustre: server umount lustre-MDT0001 complete [ 4123.989915] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4125.470172] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 4127.050738] LustreError: 290851:0:(obd_class.h:479:obd_check_dev()) Device 2 not setup [ 4127.052698] LustreError: 290851:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 4127.074703] Lustre: server umount MGS complete [ 4130.823442] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4130.974708] LustreError: 291486:0:(qmt_pool.c:1523:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [ 4130.995916] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4132.202937] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 4135.058285] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4135.182900] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4136.476839] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 4138.193128] Lustre: DEBUG MARKER: oleg429-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 4139.414079] Lustre: DEBUG MARKER: oleg429-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 4141.595293] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4143.561353] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 4145.259510] Lustre: DEBUG MARKER: oleg429-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4181.987280] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4181.987873] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4181.988476] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4181.988481] Lustre: Skipped 2 previous similar messages [ 4181.991291] Lustre: Skipped 1 previous similar message [ 4181.999996] LustreError: Skipped 1 previous similar message [ 4183.344073] LustreError: 294154:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 4183.345960] LustreError: 294154:0:(obd_class.h:479:obd_check_dev()) Skipped 1 previous similar message [ 4183.379032] Lustre: server umount lustre-OST0000 complete [ 4186.592820] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4186.594841] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4190.756364] Lustre: server umount lustre-MDT0000 complete [ 4192.123549] LustreError: 291454:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1767038685 with bad export cookie 13945889275545176882 [ 4192.130264] LustreError: MGC192.168.204.129@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4192.131554] LustreError: 291454:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 8 previous similar messages [ 4192.224298] LustreError: 294558:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [ 4192.227300] LustreError: 294558:0:(obd_class.h:479:obd_check_dev()) Skipped 9 previous similar messages [ 4192.302669] Lustre: server umount lustre-MDT0001 complete [ 4196.018831] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing unload_modules_local [ 4197.202579] Key type lgssc unregistered [ 4197.370464] LNet: 295088:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4197.374067] LNetError: 295088:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4197.381471] LNet: Removed LNI 192.168.204.129@tcp [ 4197.735749] Key type .llcrypt unregistered [ 4197.737314] Key type ._llcrypt unregistered [ 4204.544750] Key type ._llcrypt registered [ 4204.545808] Key type .llcrypt registered [ 4204.581473] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_hostid [ 4210.342791] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing load_modules_local [ 4210.664796] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4210.722959] alg: No test for adler32 (adler32-zlib) [ 4211.574357] Lustre: Lustre: Build Version: 2.17.0_RC3_1_ga0d6154 [ 4211.657261] LNet: Added LNI 192.168.204.129@tcp [8/256/0/180] [ 4213.232148] Key type lgssc registered [ 4213.614060] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4216.427035] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4218.363152] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4221.345357] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing load_modules_local [ 4223.958339] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4226.273571] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4226.293347] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4226.297435] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4227.381050] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 4227.392244] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 4227.435190] Lustre: lustre-MDT0000: new disk, initializing [ 4227.461291] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4227.467385] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 4228.758802] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 4232.740552] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4232.771686] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4232.870843] Lustre: lustre-OST0000: new disk, initializing [ 4232.872767] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 4232.875242] Lustre: Skipped 1 previous similar message [ 4232.899650] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4233.206585] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 4233.208942] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 4233.216807] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 4234.778076] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 4412.385195] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4412.388643] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4413.762293] LustreError: 299993:0:(obd_class.h:479:obd_check_dev()) Device 13 not setup [ 4413.839344] Lustre: server umount lustre-MDT0000 complete [ 4415.013100] LustreError: 298250:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1767038907 with bad export cookie 10022852242125477547 [ 4415.015168] LustreError: MGC192.168.204.129@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4415.019437] LustreError: 298250:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 4415.043162] LustreError: 300197:0:(obd_class.h:479:obd_check_dev()) Device 2 not setup [ 4415.044781] LustreError: 300197:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 4415.078707] Lustre: server umount lustre-OST0000 complete [ 4422.866184] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_hostid [ 4426.156987] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing load_modules_local [ 4429.576302] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4431.661440] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4433.641640] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4435.718513] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4437.916920] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4437.935721] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4438.018270] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 4438.030346] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 4438.072899] Lustre: lustre-MDT0000: new disk, initializing [ 4438.096205] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4438.104494] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 4439.399393] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 4443.366118] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4443.385648] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4443.417751] Lustre: 303232:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 4443.439283] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 4443.441027] Lustre: Skipped 1 previous similar message [ 4443.477803] Lustre: lustre-MDT0001: new disk, initializing [ 4443.508684] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4443.520085] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 4443.522843] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 4444.749860] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 4447.722568] Lustre: DEBUG MARKER: oleg429-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 4448.833076] Lustre: DEBUG MARKER: oleg429-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 4451.324635] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4451.355820] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4451.458543] Lustre: lustre-OST0000: new disk, initializing [ 4451.460477] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 4451.485906] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4453.327902] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 4453.942770] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 4453.945729] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 4453.975179] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 4456.263801] Lustre: DEBUG MARKER: oleg429-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4457.531198] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 4457.604262] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 4458.870917] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 4458.939469] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 4464.100832] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4464.102122] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4464.103476] LustreError: Skipped 1 previous similar message [ 4464.107290] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4464.110102] Lustre: Skipped 1 previous similar message [ 4465.585180] LustreError: 306171:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 4465.587216] LustreError: 306171:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 4465.642712] Lustre: server umount lustre-OST0000 complete [ 4469.219131] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4469.219669] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4469.223564] Lustre: Skipped 1 previous similar message [ 4469.224125] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4469.230012] Lustre: Skipped 1 previous similar message [ 4473.151277] LustreError: 306372:0:(obd_class.h:479:obd_check_dev()) Device 22 not setup [ 4473.153168] LustreError: 306372:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 4473.183600] Lustre: server umount lustre-MDT0000 complete [ 4474.476503] LustreError: 303225:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1767038967 with bad export cookie 10022852242128418324 [ 4474.480686] LustreError: 303225:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 4474.481830] LustreError: MGC192.168.204.129@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4474.610158] Lustre: server umount lustre-MDT0001 complete [ 4475.822440] Lustre: DEBUG MARKER: SKIP: conf-sanity test_111 skipping SLOW test 111 [ 4476.473951] Lustre: DEBUG MARKER: == conf-sanity test 112a: mount OST with no_create option ========================================================== 15:09:28 (1767038968) [ 4478.690760] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4478.868908] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4480.113575] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 4482.894913] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4484.331219] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 4486.078293] Lustre: DEBUG MARKER: oleg429-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 4487.250299] Lustre: DEBUG MARKER: oleg429-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 4489.535811] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4491.571931] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 4493.497687] Lustre: DEBUG MARKER: oleg429-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4496.133502] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4496.158563] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4496.218389] Lustre: lustre-OST0001: new disk, initializing [ 4496.220674] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 4496.254793] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 4496.257100] Lustre: Skipped 2 previous similar messages [ 4497.666098] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 4497.669045] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 4497.686184] LustreError: 309842:0:(osp_precreate.c:969:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [ 4497.689576] LustreError: 309842:0:(osp_precreate.c:969:osp_precreate_cleanup_orphans()) Skipped 1 previous similar message [ 4498.383669] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 4498.727165] LustreError: 309846:0:(osp_precreate.c:969:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [ 4498.730182] LustreError: 309846:0:(osp_precreate.c:969:osp_precreate_cleanup_orphans()) Skipped 1 previous similar message [ 4499.749565] LustreError: 309846:0:(osp_precreate.c:969:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [ 4499.752402] LustreError: 309846:0:(osp_precreate.c:969:osp_precreate_cleanup_orphans()) Skipped 1 previous similar message [ 4501.797269] LustreError: 309846:0:(osp_precreate.c:969:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [ 4501.801535] LustreError: 309846:0:(osp_precreate.c:969:osp_precreate_cleanup_orphans()) Skipped 3 previous similar messages [ 4502.415497] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 4504.522347] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 2 sec [ 4505.798916] Lustre: DEBUG MARKER: oleg429-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff8a61e0a02000.ost_server_uuid 50 [ 4505.891666] LustreError: 309842:0:(osp_precreate.c:969:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [ 4505.894455] LustreError: 309842:0:(osp_precreate.c:969:osp_precreate_cleanup_orphans()) Skipped 7 previous similar messages [ 4506.416097] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff8a61e0a02000.ost_server_uuid in FULL state after 0 sec [ 4507.976198] Lustre: DEBUG MARKER: oleg429-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff8a61e0a02000.ost_server_uuid 50 [ 4508.542428] Lustre: DEBUG MARKER: osc.lustre-OST0001-osc-ffff8a61e0a02000.ost_server_uuid in FULL state after 0 sec [ 4509.994026] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 4520.417279] LustreError: lustre-OST0001-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4520.417361] Lustre: lustre-OST0001-osc-MDT0001: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4520.419556] LustreError: Skipped 1 previous similar message [ 4520.422975] Lustre: Skipped 1 previous similar message [ 4520.425505] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 4520.429161] Lustre: Skipped 1 previous similar message [ 4524.517989] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 4524.522541] Lustre: Skipped 2 previous similar messages [ 4526.318564] LustreError: 310969:0:(obd_class.h:479:obd_check_dev()) Device 25 not setup [ 4526.321530] LustreError: 310969:0:(obd_class.h:479:obd_check_dev()) Skipped 13 previous similar messages [ 4526.345124] Lustre: server umount lustre-OST0001 complete [ 4526.849581] LustreError: 308745:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 192.168.204.29@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4526.853615] LustreError: 308745:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 4 previous similar messages [ 4528.609157] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4528.611258] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4529.639245] LustreError: 310973:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4529.644671] LustreError: 310973:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 4534.042045] Lustre: server umount lustre-OST0000 complete [ 4539.361050] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4539.364267] LustreError: Skipped 1 previous similar message [ 4539.366041] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4539.370976] Lustre: Skipped 1 previous similar message [ 4539.372934] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4539.374786] Lustre: Skipped 3 previous similar messages [ 4541.442318] LustreError: 311374:0:(obd_class.h:479:obd_check_dev()) Device 28 not setup [ 4541.445736] LustreError: 311374:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 4541.474114] Lustre: server umount lustre-MDT0000 complete [ 4542.722129] LustreError: 307126:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1767039035 with bad export cookie 10022852242128419192 [ 4542.725300] LustreError: 307126:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 4542.726343] LustreError: MGC192.168.204.129@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4542.841252] Lustre: server umount lustre-MDT0001 complete [ 4546.258882] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing unload_modules_local [ 4547.235605] Key type lgssc unregistered [ 4547.369358] LNet: 312107:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4547.371765] LNetError: 312107:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4547.379325] LNet: Removed LNI 192.168.204.129@tcp [ 4547.654481] Key type .llcrypt unregistered [ 4547.655450] Key type ._llcrypt unregistered [ 4556.449225] Key type ._llcrypt registered [ 4556.450262] Key type .llcrypt registered [ 4556.487836] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing load_modules_local [ 4556.803269] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4556.819699] alg: No test for adler32 (adler32-zlib) [ 4557.664163] Lustre: Lustre: Build Version: 2.17.0_RC3_1_ga0d6154 [ 4557.758810] LNet: Added LNI 192.168.204.129@tcp [8/256/0/180] [ 4559.336169] Key type lgssc registered [ 4559.656528] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4562.068225] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4562.073377] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4563.229775] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4564.464205] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 4566.117446] Lustre: DEBUG MARKER: oleg429-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 4568.382883] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4568.523647] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4569.913655] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 4572.873336] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4572.986040] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4573.988125] LustreError: 315036:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4573.992456] LustreError: 315036:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 4573.993032] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:65) [ 4574.721405] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 4576.366372] Lustre: DEBUG MARKER: oleg429-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4578.482167] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4578.549996] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 4578.580710] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:33) [ 4580.345726] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 4582.087790] Lustre: DEBUG MARKER: oleg429-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid [ 4584.575958] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing wait_import_state (FULL|IDLE) os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [ 4584.660254] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 4660.705661] LustreError: lustre-OST0001-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4660.707974] Lustre: lustre-OST0001-osc-MDT0000: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4660.712319] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 4661.730401] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 4661.734371] Lustre: lustre-OST0001-osc-MDT0001: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4661.735173] LustreError: lustre-OST0001-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4666.415136] LustreError: 316903:0:(obd_class.h:479:obd_check_dev()) Device 27 not setup [ 4666.434137] Lustre: server umount lustre-OST0001 complete [ 4666.854217] LustreError: 315223:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4666.858162] LustreError: 315223:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 4671.457139] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4671.459251] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4671.462758] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4671.464409] Lustre: Skipped 1 previous similar message [ 4671.972731] LustreError: 315035:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4671.980517] LustreError: 315035:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 4674.158095] LustreError: 317105:0:(obd_class.h:479:obd_check_dev()) Device 23 not setup [ 4674.160497] LustreError: 317105:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 4674.200752] Lustre: server umount lustre-OST0000 complete [ 4676.065460] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4676.070284] LustreError: Skipped 1 previous similar message [ 4676.073410] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4676.081380] Lustre: Skipped 1 previous similar message [ 4676.084917] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4676.089027] Lustre: Skipped 1 previous similar message [ 4681.663969] LustreError: 317306:0:(obd_class.h:479:obd_check_dev()) Device 10 not setup [ 4681.665944] LustreError: 317306:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 4681.692524] Lustre: server umount lustre-MDT0000 complete [ 4682.209042] LustreError: 313484:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4682.213392] LustreError: 313484:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 4682.891199] LustreError: 316798:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1767039175 with bad export cookie 616806813232887980 [ 4682.895642] LustreError: MGC192.168.204.129@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4682.896311] LustreError: 316798:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 4683.019952] Lustre: server umount lustre-MDT0001 complete [ 4686.208586] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing unload_modules_local [ 4687.192716] Key type lgssc unregistered [ 4687.342679] LNet: 318040:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4687.344973] LNetError: 318040:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4687.353650] LNet: Removed LNI 192.168.204.129@tcp [ 4687.689855] Key type .llcrypt unregistered [ 4687.691624] Key type ._llcrypt unregistered [ 4705.186890] Key type ._llcrypt registered [ 4705.187866] Key type .llcrypt registered [ 4705.225808] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing load_modules_local [ 4705.652185] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4705.667505] alg: No test for adler32 (adler32-zlib) [ 4706.518888] Lustre: Lustre: Build Version: 2.17.0_RC3_1_ga0d6154 [ 4706.604387] LNet: Added LNI 192.168.204.129@tcp [8/256/0/180] [ 4708.184113] Key type lgssc registered [ 4708.551136] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4711.592796] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4711.600497] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4712.766689] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4714.074695] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 4716.900620] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4717.042312] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4718.337082] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 4719.260056] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 4721.404925] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4721.534058] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4722.533810] LustreError: 321926:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4722.537056] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:97) [ 4722.539157] LustreError: 321926:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 4723.420292] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 4726.270515] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4726.345697] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 4727.409606] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:65) [ 4728.315189] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 4732.406466] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 4740.060082] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 4752.865025] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4752.868647] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4752.872105] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4753.379166] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4753.379812] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4753.383522] Lustre: Skipped 1 previous similar message [ 4754.560913] LustreError: 323698:0:(obd_class.h:479:obd_check_dev()) Device 10 not setup [ 4754.587892] Lustre: server umount lustre-MDT0000 complete [ 4755.723354] LustreError: 320264:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1767039248 with bad export cookie 5795670572910699578 [ 4755.726526] LustreError: 320264:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4755.726891] LustreError: MGC192.168.204.129@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4755.796706] LustreError: 323899:0:(obd_class.h:479:obd_check_dev()) Device 18 not setup [ 4755.799814] LustreError: 323899:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 4755.846373] Lustre: server umount lustre-MDT0001 complete [ 4767.732066] LustreError: 324099:0:(obd_class.h:479:obd_check_dev()) Device 23 not setup [ 4767.733685] LustreError: 324099:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 4767.748622] Lustre: server umount lustre-OST0000 complete [ 4778.162266] LustreError: 324301:0:(obd_class.h:479:obd_check_dev()) Device 2 not setup [ 4778.163918] LustreError: 324301:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 4778.204173] Lustre: server umount lustre-OST0001 complete [ 4783.189818] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing load_modules_local [ 4786.458933] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4786.597757] LustreError: 325475:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4786.616700] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4787.871791] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 4790.661945] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4792.020320] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 4792.950879] Lustre: 326584:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 4795.099619] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4795.244340] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4795.246313] Lustre: Skipped 1 previous similar message [ 4796.263971] LustreError: 326941:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4796.271613] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:129) [ 4797.225757] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 4799.978483] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4801.148847] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:97) [ 4802.028397] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 4805.123946] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 4806.360262] Lustre: 328428:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 4816.353035] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4816.354911] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4816.357968] Lustre: Skipped 1 previous similar message [ 4816.359561] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4816.361343] Lustre: Skipped 3 previous similar messages [ 4818.439662] LustreError: 328533:0:(obd_class.h:479:obd_check_dev()) Device 10 not setup [ 4818.442937] LustreError: 328533:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 4818.471870] Lustre: server umount lustre-MDT0000 complete [ 4819.667399] LustreError: 326203:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1767039312 with bad export cookie 5795670572910701272 [ 4819.671518] LustreError: 326203:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 4819.672016] LustreError: MGC192.168.204.129@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4819.784453] Lustre: server umount lustre-MDT0001 complete [ 4830.134100] LustreError: 328935:0:(obd_class.h:479:obd_check_dev()) Device 23 not setup [ 4830.135613] LustreError: 328935:0:(obd_class.h:479:obd_check_dev()) Skipped 15 previous similar messages [ 4830.146643] Lustre: server umount lustre-OST0000 complete [ 4841.689354] Lustre: server umount lustre-OST0001 complete [ 4842.808536] Lustre: DEBUG MARKER: SKIP: conf-sanity test_114 skipping SLOW test 114 [ 4843.296699] Lustre: DEBUG MARKER: SKIP: conf-sanity test_115 skipping excluded test 115 [ 4843.869270] Lustre: DEBUG MARKER: == conf-sanity test 116: big size MDT support ============ 15:15:36 (1767039336) [ 4851.366947] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing load_modules_local [ 4854.701852] loop0: detected capacity change from 0 to 1099511627776 [ 4854.846844] SGI XFS with ACLs, security attributes, quota, no debug enabled [ 4854.849168] *** VALIDATE xfs *** [ 4854.853309] XFS (loop0): Mounting V5 Filesystem [ 4854.860765] XFS (loop0): Ending clean mount [ 4855.367846] loop1: detected capacity change from 0 to 18691697672192 [ 4869.832770] LDISKFS-fs (loop1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4876.331891] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_hostid [ 4879.367959] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing load_modules_local [ 4882.840921] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4885.070233] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4886.933496] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4889.855959] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4892.400706] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4892.425823] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4892.510357] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 4892.520901] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 4892.556470] Lustre: lustre-MDT0000: new disk, initializing [ 4892.581080] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4892.583016] Lustre: Skipped 1 previous similar message [ 4892.588462] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 4893.851258] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 4897.865127] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4897.890305] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4897.925110] Lustre: 333503:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 4897.946562] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 4897.948053] Lustre: Skipped 1 previous similar message [ 4897.981946] Lustre: lustre-MDT0001: new disk, initializing [ 4898.023980] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 4898.028220] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 4899.235141] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 4902.504368] Lustre: DEBUG MARKER: oleg429-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 4903.865210] Lustre: DEBUG MARKER: oleg429-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 4906.163445] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4906.185696] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4906.302302] Lustre: lustre-OST0000: new disk, initializing [ 4906.304296] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 4907.448138] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 4907.453774] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 4907.489239] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 4908.162863] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 4911.130587] Lustre: DEBUG MARKER: oleg429-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4912.554260] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 4912.659686] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 4914.045086] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 4914.123540] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 4917.731160] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4917.731316] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4917.734112] LustreError: Skipped 1 previous similar message [ 4917.739916] Lustre: Skipped 3 previous similar messages [ 4917.742949] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4917.747370] Lustre: Skipped 4 previous similar messages [ 4920.882107] LustreError: 336444:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 4920.884937] LustreError: 336444:0:(obd_class.h:479:obd_check_dev()) Skipped 9 previous similar messages [ 4920.940499] Lustre: server umount lustre-OST0000 complete [ 4922.849838] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4922.850550] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4922.853155] Lustre: Skipped 1 previous similar message [ 4928.728483] Lustre: server umount lustre-MDT0000 complete [ 4929.967567] LustreError: 333494:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1767039422 with bad export cookie 5795670572910702826 [ 4929.970785] LustreError: MGC192.168.204.129@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4929.971084] LustreError: 333494:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 4930.096597] Lustre: server umount lustre-MDT0001 complete [ 4932.820131] XFS (loop0): Unmounting Filesystem [ 4934.809604] Lustre: DEBUG MARKER: == conf-sanity test 117: lctl get_param return errors properly ========================================================== 15:17:06 (1767039426) [ 4937.511364] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4937.676179] LustreError: 337662:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4937.703390] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4937.705581] Lustre: Skipped 2 previous similar messages [ 4939.020872] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 4942.058522] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4943.461781] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 4945.182469] Lustre: DEBUG MARKER: oleg429-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 4946.323407] Lustre: DEBUG MARKER: oleg429-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 4948.667736] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4950.670465] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 4952.477180] Lustre: DEBUG MARKER: oleg429-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4973.538692] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4973.539509] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4973.540609] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4973.540613] Lustre: Skipped 3 previous similar messages [ 4973.545104] Lustre: Skipped 1 previous similar message [ 4973.554783] LustreError: Skipped 1 previous similar message [ 4977.326396] LustreError: 340026:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 4977.328204] LustreError: 340026:0:(obd_class.h:479:obd_check_dev()) Skipped 17 previous similar messages [ 4977.361509] Lustre: server umount lustre-OST0000 complete [ 4983.265206] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4984.682403] Lustre: server umount lustre-MDT0000 complete [ 4985.983158] LustreError: 337642:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1767039478 with bad export cookie 5795670572910703708 [ 4985.985802] LustreError: MGC192.168.204.129@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4985.985847] LustreError: 337642:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 4986.107457] Lustre: server umount lustre-MDT0001 complete [ 4989.501274] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing unload_modules_local [ 4990.443382] Key type lgssc unregistered [ 4990.567313] LNet: 340961:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4990.571633] LNetError: 340961:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4990.579339] LNet: Removed LNI 192.168.204.129@tcp [ 4990.904146] Key type .llcrypt unregistered [ 4990.905555] Key type ._llcrypt unregistered [ 5000.340962] Key type ._llcrypt registered [ 5000.341970] Key type .llcrypt registered [ 5000.380538] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing load_modules_local [ 5000.806438] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5000.816952] alg: No test for adler32 (adler32-zlib) [ 5001.663369] Lustre: Lustre: Build Version: 2.17.0_RC3_1_ga0d6154 [ 5001.741332] LNet: Added LNI 192.168.204.129@tcp [8/256/0/180] [ 5003.320131] Key type lgssc registered [ 5003.699169] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5006.261170] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5006.266814] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5007.426274] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5008.730964] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 5011.533686] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5011.659696] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5012.890341] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 5014.640986] Lustre: DEBUG MARKER: oleg429-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 5015.772212] Lustre: DEBUG MARKER: oleg429-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 5017.909064] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5018.016546] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5019.828935] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 5021.600954] Lustre: DEBUG MARKER: oleg429-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 5027.298269] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [ 5027.298768] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5027.301071] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 5027.305401] Lustre: Skipped 1 previous similar message [ 5027.308154] Lustre: Skipped 1 previous similar message [ 5028.103283] Lustre: lustre-MDT0001: Not available for connect from 192.168.204.29@tcp (stopping) [ 5031.165403] LustreError: 344777:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [ 5031.186644] Lustre: server umount lustre-MDT0001 complete [ 5032.095123] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5032.418061] LustreError: 342488:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5032.426388] LustreError: 342488:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 5033.216487] LustreError: 344104:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.204.29@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5034.535180] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5034.565876] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [ 5034.567411] Lustre: Found index 1 for lustre-MDT0001, updating log [ 5034.579520] Lustre: 342478:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 5034.692183] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5035.956697] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 5040.102688] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 5042.150823] LustreError: 345911:0:(obd_mount.c:188:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [ 5042.154904] LustreError: 345911:0:(tgt_mount.c:669:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [ 5042.159118] LustreError: 342481:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [ 5042.173552] LustreError: 345914:0:(obd_config.c:2001:class_config_llog_handler()) MGC192.168.204.129@tcp: cfg command failed: rc = -17 [ 5042.175935] Lustre: cmd=cf001 0:lustre-MDT0001-osp-MDT0000 1:osp 2:lustre-MDT0000-mdtlov_UUID [ 5042.175935] [ 5340.421750] Lustre: lustre-MDT0001: Not available for connect from 192.168.204.29@tcp (stopping) [ 5342.178401] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5342.179352] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [ 5344.899755] LustreError: 357451:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [ 5344.902845] LustreError: 357451:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 5344.927711] Lustre: server umount lustre-MDT0001 complete [ 5345.536715] LustreError: 342483:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.204.29@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5345.883573] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5348.457056] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5348.494461] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [ 5348.496113] Lustre: Found index 1 for lustre-MDT0001, updating log [ 5348.510988] Lustre: 342479:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 5348.647426] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5350.052350] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 5353.963836] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 5358.056127] LustreError: 358676:0:(obd_mount.c:188:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [ 5358.059093] LustreError: 358676:0:(tgt_mount.c:669:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [ 5358.061977] LustreError: 342481:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [ 5358.065424] LustreError: 342481:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [ 5358.084489] LustreError: 358679:0:(obd_config.c:689:class_setup()) Device 8 already setup (type osp) [ 5358.087158] LustreError: 358679:0:(obd_config.c:797:class_detach()) OBD device 8 still set up [ 5358.089023] LustreError: 358679:0:(obd_config.c:2001:class_config_llog_handler()) MGC192.168.204.129@tcp: cfg command failed: rc = -17 [ 5358.091567] Lustre: cmd=cf003 0:lustre-MDT0001-osp-MDT0000 1:lustre-MDT0001_UUID 2:192.168.204.129@tcp [ 5358.091567] [ 5652.741776] Lustre: lustre-MDT0001: Not available for connect from 192.168.204.29@tcp (stopping) [ 5652.743666] Lustre: Skipped 2 previous similar messages [ 5656.035496] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5656.035888] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [ 5656.041860] Lustre: Skipped 1 previous similar message [ 5657.856351] Lustre: lustre-MDT0001: Not available for connect from 192.168.204.29@tcp (stopping) [ 5657.858606] Lustre: Skipped 2 previous similar messages [ 5666.272111] Lustre: lustre-MDT0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 5666.273458] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 5666.277469] Lustre: Skipped 4 previous similar messages [ 5666.310461] LustreError: 370117:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [ 5666.313369] LustreError: 370117:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 5666.337956] Lustre: server umount lustre-MDT0001 complete [ 5667.261052] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5668.096436] LustreError: 346016:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.204.29@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5668.103821] LustreError: 346016:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 5669.705860] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5669.735169] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [ 5669.736581] Lustre: Found index 1 for lustre-MDT0001, updating log [ 5669.750585] Lustre: 342479:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 5669.863477] Lustre: lustre-MDT0001: Imperative Recovery enabled, recovery window shrunk from 60-180 down to 60-180 [ 5671.172221] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 5674.983221] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 5679.592063] LustreError: 371346:0:(obd_mount.c:188:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [ 5679.594081] LustreError: 371346:0:(tgt_mount.c:669:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [ 5679.596277] LustreError: 342481:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [ 5679.598234] LustreError: 342481:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [ 5679.616611] LustreError: 371349:0:(lod_lov.c:212:lod_add_device()) lustre-MDT0001-osp-MDT0000: cannot connect to next dev lustre-MDT0001_UUID (-114) [ 5679.621889] LustreError: 371349:0:(obd_config.c:2001:class_config_llog_handler()) MGC192.168.204.129@tcp: cfg command failed: rc = -114 [ 5679.624814] Lustre: cmd=cf014 0:lustre-MDT0000-mdtlov 1:lustre-MDT0001_UUID 2:1 3:1 [ 5679.624814] [ 5987.297850] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5987.299711] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 5987.299927] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5987.303651] Lustre: Skipped 2 previous similar messages [ 5991.037217] LustreError: 382953:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup [ 5991.039481] LustreError: 382953:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 5991.065567] Lustre: server umount lustre-MDT0000 complete [ 5992.172903] LustreError: 345348:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1767040485 with bad export cookie 9070988759656405519 [ 5992.175608] LustreError: 345348:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 5992.175890] LustreError: MGC192.168.204.129@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5992.416901] LustreError: 342483:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5992.417232] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5998.270159] LustreError: 383154:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [ 5998.272294] LustreError: 383154:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 5998.323819] Lustre: server umount lustre-MDT0001 complete [ 5999.577174] Lustre: server umount lustre-OST0000 complete [ 6002.381317] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_hostid [ 6005.114518] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing load_modules_local [ 6008.166218] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6010.127954] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6012.062837] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6014.048238] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6016.094612] Lustre: DEBUG MARKER: == conf-sanity test 120: cross-target rename should not create bad symlinks ========================================================== 15:35:08 (1767040508) [ 6018.199927] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6018.222541] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6018.303545] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 6018.316617] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 6018.363370] Lustre: lustre-MDT0000: new disk, initializing [ 6018.389302] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6018.398244] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 6019.630713] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 6023.555646] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6023.579205] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 6023.611022] Lustre: 385886:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 6023.631697] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 6023.633797] Lustre: Skipped 1 previous similar message [ 6023.666977] Lustre: lustre-MDT0001: new disk, initializing [ 6023.699588] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 6023.709756] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 6023.712510] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 6024.875217] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 6027.766367] Lustre: DEBUG MARKER: oleg429-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 6028.894888] Lustre: DEBUG MARKER: oleg429-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 6031.139078] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6031.164298] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 6031.274347] Lustre: lustre-OST0000: new disk, initializing [ 6031.276193] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 6031.308763] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 6033.137719] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_default_debug -1 all [ 6033.980405] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 6033.983951] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 6034.021499] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 6036.114039] Lustre: DEBUG MARKER: oleg429-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 6039.010270] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 6039.011288] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6039.014380] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6039.016856] Lustre: Skipped 2 previous similar messages [ 6039.022156] Lustre: Skipped 6 previous similar messages [ 6044.802374] LustreError: 388329:0:(obd_class.h:479:obd_check_dev()) Device 22 not setup [ 6044.804023] LustreError: 388329:0:(obd_class.h:479:obd_check_dev()) Skipped 11 previous similar messages [ 6044.838126] Lustre: server umount lustre-MDT0000 complete [ 6046.006101] LustreError: 385877:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1767040538 with bad export cookie 9070988759656407458 [ 6046.009662] LustreError: 385877:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 6046.009857] LustreError: 386678:0:(osp_object.c:617:osp_attr_get()) lustre-MDT0000-osp-MDT0001: osp_attr_get update error [0x200000401:0x2:0x0]: rc = -5 [ 6046.010030] LustreError: MGC192.168.204.129@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6046.127158] Lustre: server umount lustre-MDT0001 complete [ 6057.569271] Lustre: server umount lustre-OST0000 complete [ 6066.385166] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing set_hostid [ 6069.479400] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing load_modules_local [ 6072.712914] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6074.825048] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6076.841524] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6079.010465] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6079.549712] Lustre: DEBUG MARKER: == conf-sanity test complete, duration 5905 sec ========== 15:36:11 (1767040571) [ 6080.090337] Lustre: DEBUG MARKER: === conf-sanity: start cleanup 15:36:12 (1767040572) === [ 6081.229455] Lustre: DEBUG MARKER: === conf-sanity: finish cleanup 15:36:13 (1767040573) === [ 6094.399349] Lustre: DEBUG MARKER: oleg429-server.virtnet: executing unload_modules_local [ 6095.393741] Key type lgssc unregistered [ 6095.519403] LNet: 393833:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 6095.521749] LNetError: 393833:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 6095.531392] LNet: Removed LNI 192.168.204.129@tcp [ 6095.802867] Key type .llcrypt unregistered [ 6095.803807] Key type ._llcrypt unregistered