[ 0.000000] Linux version 4.18.0rh8.10-debug (green@maintenance) (gcc version 8.5.0 20210514 (Red Hat 8.5.0-26) (GCC)) #2 SMP Mon Jul 14 01:24:22 EDT 2025 [ 0.000000] Command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers' [ 0.000000] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256 [ 0.000000] x86/fpu: Enabled xstate features 0x7, context size is 832 bytes, using 'standard' format. [ 0.000000] signal: max sigframe size: 1776 [ 0.000000] BIOS-provided physical RAM map: [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x00000000bffcdfff] usable [ 0.000000] BIOS-e820: [mem 0x00000000bffce000-0x00000000bfffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x0000000146dfffff] usable [ 0.000000] NX (Execute Disable) protection: active [ 0.000000] SMBIOS 2.8 present. [ 0.000000] DMI: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.16.2-1.fc38 04/01/2014 [ 0.000000] Hypervisor detected: KVM [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00 [ 0.000000] kvm-clock: using sched offset of 508471445 cycles [ 0.000000] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns [ 0.000000] tsc: Detected 2399.998 MHz processor [ 0.000000] last_pfn = 0x146e00 max_arch_pfn = 0x400000000 [ 0.000000] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT [ 0.000000] last_pfn = 0xbffce max_arch_pfn = 0x400000000 [ 0.000000] found SMP MP-table at [mem 0x000f5b30-0x000f5b3f] [ 0.000000] RAMDISK: [mem 0xbcc54000-0xbffbffff] [ 0.000000] ACPI: Early table checksum verification disabled [ 0.000000] ACPI: RSDP 0x00000000000F5950 000014 (v00 BOCHS ) [ 0.000000] ACPI: RSDT 0x00000000BFFE1BB7 000034 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACP 0x00000000BFFE1A53 000074 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: DSDT 0x00000000BFFE0040 001A13 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACS 0x00000000BFFE0000 000040 [ 0.000000] ACPI: APIC 0x00000000BFFE1AC7 000090 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: HPET 0x00000000BFFE1B57 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: WAET 0x00000000BFFE1B8F 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: Reserving FACP table memory at [mem 0xbffe1a53-0xbffe1ac6] [ 0.000000] ACPI: Reserving DSDT table memory at [mem 0xbffe0040-0xbffe1a52] [ 0.000000] ACPI: Reserving FACS table memory at [mem 0xbffe0000-0xbffe003f] [ 0.000000] ACPI: Reserving APIC table memory at [mem 0xbffe1ac7-0xbffe1b56] [ 0.000000] ACPI: Reserving HPET table memory at [mem 0xbffe1b57-0xbffe1b8e] [ 0.000000] ACPI: Reserving WAET table memory at [mem 0xbffe1b8f-0xbffe1bb6] [ 0.000000] No NUMA configuration found [ 0.000000] Faking a node at [mem 0x0000000000000000-0x0000000146dfffff] [ 0.000000] NODE_DATA(0) allocated [mem 0x1465a3000-0x1465cdfff] [ 0.000000] Reserving 256MB of memory at 2752MB for crashkernel (System RAM: 4205MB) [ 0.000000] Zone ranges: [ 0.000000] DMA [mem 0x0000000000001000-0x0000000000ffffff] [ 0.000000] DMA32 [mem 0x0000000001000000-0x00000000ffffffff] [ 0.000000] Normal [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Device empty [ 0.000000] Movable zone start for each node [ 0.000000] Early memory node ranges [ 0.000000] node 0: [mem 0x0000000000001000-0x000000000009efff] [ 0.000000] node 0: [mem 0x0000000000100000-0x00000000bffcdfff] [ 0.000000] node 0: [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Zeroed struct page in unavailable ranges: 4756 pages [ 0.000000] Initmem setup node 0 [mem 0x0000000000001000-0x0000000146dfffff] [ 0.000000] ACPI: PM-Timer IO Port: 0x608 [ 0.000000] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1]) [ 0.000000] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23 [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level) [ 0.000000] Using ACPI (MADT) for SMP configuration information [ 0.000000] ACPI: HPET id: 0x8086a201 base: 0xfed00000 [ 0.000000] TSC deadline timer available [ 0.000000] smpboot: Allowing 4 CPUs, 0 hotplug CPUs [ 0.000000] kvm-guest: KVM setup pv remote TLB flush [ 0.000000] kvm-guest: setup PV sched yield [ 0.000000] PM: Registered nosave memory: [mem 0x00000000-0x00000fff] [ 0.000000] PM: Registered nosave memory: [mem 0x0009f000-0x0009ffff] [ 0.000000] PM: Registered nosave memory: [mem 0x000a0000-0x000effff] [ 0.000000] PM: Registered nosave memory: [mem 0x000f0000-0x000fffff] [ 0.000000] PM: Registered nosave memory: [mem 0xbffce000-0xbfffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xc0000000-0xfeffbfff] [ 0.000000] PM: Registered nosave memory: [mem 0xfeffc000-0xfeffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xff000000-0xfffbffff] [ 0.000000] PM: Registered nosave memory: [mem 0xfffc0000-0xffffffff] [ 0.000000] [mem 0xc0000000-0xfeffbfff] available for PCI devices [ 0.000000] Booting paravirtualized kernel on KVM [ 0.000000] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns [ 0.000000] setup_percpu: NR_CPUS:8192 nr_cpumask_bits:4 nr_cpu_ids:4 nr_node_ids:1 [ 0.000000] percpu: Embedded 63 pages/cpu s221184 r8192 d28672 u524288 [ 0.000000] kvm-guest: PV spinlocks enabled [ 0.000000] PV qspinlock hash table entries: 256 (order: 0, 4096 bytes, linear) [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 1059606 [ 0.000000] Policy zone: Normal [ 0.000000] Kernel command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] Specific versions of hardware are certified with Red Hat Enterprise Linux 8. Please see the list of hardware certified with Red Hat Enterprise Linux 8 at https://catalog.redhat.com. [ 0.000000] audit: disabled (until reboot) [ 0.000000] software IO TLB: area num 4. [ 0.000000] Memory: 2829652K/4306352K available (18435K kernel code, 11221K rwdata, 7248K rodata, 2908K init, 18040K bss, 524580K reserved, 0K cma-reserved) [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1 [ 0.000000] kmemleak: Kernel memory leak detector disabled [ 0.000000] ftrace: allocating 41240 entries in 162 pages [ 0.000000] ftrace: allocated 162 pages with 3 groups [ 0.000000] rcu: Hierarchical RCU implementation. [ 0.000000] rcu: RCU event tracing is enabled. [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=8192 to nr_cpu_ids=4. [ 0.000000] rcu: RCU callback double-/use-after-free debug enabled. [ 0.000000] Rude variant of Tasks RCU enabled. [ 0.000000] Tracing variant of Tasks RCU enabled. [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies. [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4 [ 0.000000] NR_IRQS: 524544, nr_irqs: 456, preallocated irqs: 16 [ 0.000000] random: get_random_bytes called from start_kernel+0x622/0x9a8 with crng_init=0 [ 0.001000] Console: colour *CGA 80x25 [ 0.001000] printk: console [ttyS1] enabled [ 0.001000] ACPI: Core revision 20220331 [ 0.001000] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns [ 0.001012] APIC: Switch to symmetric I/O mode setup [ 0.002000] x2apic enabled [ 0.002009] Switched APIC routing to physical x2apic. [ 0.003013] kvm-guest: setup PV IPIs [ 0.006554] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1 [ 0.007000] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 0.007025] Calibrating delay loop (skipped) preset value.. 4799.99 BogoMIPS (lpj=2399998) [ 0.008014] pid_max: default: 32768 minimum: 301 [ 0.009126] LSM: Security Framework initializing [ 0.010064] Yama: becoming mindful. [ 0.012032] SELinux: Initializing. [ 0.013084] *** VALIDATE selinux *** [ 0.021854] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, vmalloc) [ 0.026408] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, vmalloc) [ 0.027178] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.028115] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.029111] *** VALIDATE tmpfs *** [ 0.031239] *** VALIDATE proc *** [ 0.032247] *** VALIDATE cgroup *** [ 0.033012] *** VALIDATE cgroup2 *** [ 0.034270] x86/cpu: User Mode Instruction Prevention (UMIP) activated [ 0.035168] Last level iTLB entries: 4KB 0, 2MB 0, 4MB 0 [ 0.036010] Last level dTLB entries: 4KB 0, 2MB 0, 4MB 0, 1GB 0 [ 0.037034] Spectre V2 : User space: Vulnerable [ 0.038012] Speculative Store Bypass: Vulnerable [ 0.041242] debug: unmapping init [mem 0xffffffffa4659000-0xffffffffa4660fff] [ 0.043200] smpboot: CPU0: Intel(R) Xeon(R) CPU E5-2695 v2 @ 2.40GHz (family: 0x6, model: 0x3e, stepping: 0x4) [ 0.044633] Performance Events: IvyBridge events, full-width counters, Intel PMU driver. [ 0.045032] ... version: 2 [ 0.046016] ... bit width: 48 [ 0.047024] ... generic registers: 4 [ 0.048020] ... value mask: 0000ffffffffffff [ 0.049019] ... max period: 00007fffffffffff [ 0.050022] ... fixed-purpose events: 3 [ 0.051014] ... event mask: 000000070000000f [ 0.052282] rcu: Hierarchical SRCU implementation. [ 0.054343] smp: Bringing up secondary CPUs ... [ 0.055622] x86: Booting SMP configuration: [ 0.056031] .... node #0, CPUs: #1 #2 #3 [ 0.059215] smp: Brought up 1 node, 4 CPUs [ 0.061015] smpboot: Max logical packages: 1 [ 0.062026] smpboot: Total of 4 processors activated (19199.98 BogoMIPS) [ 0.133715] node 0 deferred pages initialised in 70ms [ 0.137159] devtmpfs: initialized [ 0.138278] x86/mm: Memory block size: 128MB [ 0.140516] gcov: version magic: 0x41383552 [ 0.143305] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns [ 0.144136] futex hash table entries: 1024 (order: 4, 65536 bytes, vmalloc) [ 0.145341] pinctrl core: initialized pinctrl subsystem [ 0.146255] [ 0.146765] ************************************************************* [ 0.147016] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.148012] ** ** [ 0.149012] ** IOMMU DebugFS SUPPORT HAS BEEN ENABLED IN THIS KERNEL ** [ 0.150020] ** ** [ 0.151012] ** This means that this kernel is built to expose internal ** [ 0.152014] ** IOMMU data structures, which may compromise security on ** [ 0.153021] ** your system. ** [ 0.154016] ** ** [ 0.155018] ** If you see this message and you are not debugging the ** [ 0.156015] ** kernel, report this immediately to your vendor! ** [ 0.157016] ** ** [ 0.158020] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.159018] ************************************************************* [ 0.160651] NET: Registered protocol family 16 [ 0.161427] DMA: preallocated 512 KiB GFP_KERNEL pool for atomic allocations [ 0.162083] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations [ 0.163070] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations [ 0.164502] cpuidle: using governor menu [ 0.165000] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5 [ 0.167563] PCI: Using configuration type 1 for base access [ 0.170142] core: PMU erratum BJ122, BV98, HSD29 worked around, HT is on [ 0.180058] HugeTLB registered 1.00 GiB page size, pre-allocated 0 pages [ 0.181025] HugeTLB registered 2.00 MiB page size, pre-allocated 0 pages [ 0.183152] cryptd: max_cpu_qlen set to 1000 [ 0.185291] ACPI: Added _OSI(Module Device) [ 0.186020] ACPI: Added _OSI(Processor Device) [ 0.187018] ACPI: Added _OSI(3.0 _SCP Extensions) [ 0.188013] ACPI: Added _OSI(Processor Aggregator Device) [ 0.191433] ACPI: 1 ACPI AML tables successfully acquired and loaded [ 0.194042] ACPI: Interpreter enabled [ 0.195063] ACPI: PM: (supports S0 S3 S4 S5) [ 0.196017] ACPI: Using IOAPIC for interrupt routing [ 0.197116] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug [ 0.198445] ACPI: Enabled 2 GPEs in block 00 to 0F [ 0.207945] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff]) [ 0.208052] acpi PNP0A03:00: _OSC: OS supports [ASPM ClockPM Segments MSI HPX-Type3] [ 0.209028] acpi PNP0A03:00: _OSC: not requesting OS control; OS requires [ExtendedConfig ASPM ClockPM MSI] [ 0.210123] acpi PNP0A03:00: fail to add MMCONFIG information, can't access extended PCI configuration space under this bridge. [ 0.212335] acpiphp: Slot [2] registered [ 0.213144] acpiphp: Slot [3] registered [ 0.214098] acpiphp: Slot [4] registered [ 0.215113] acpiphp: Slot [5] registered [ 0.216166] acpiphp: Slot [6] registered [ 0.217158] acpiphp: Slot [7] registered [ 0.218110] acpiphp: Slot [8] registered [ 0.219115] acpiphp: Slot [9] registered [ 0.220121] acpiphp: Slot [10] registered [ 0.221105] acpiphp: Slot [11] registered [ 0.222215] acpiphp: Slot [12] registered [ 0.223297] acpiphp: Slot [13] registered [ 0.224185] acpiphp: Slot [14] registered [ 0.225175] acpiphp: Slot [15] registered [ 0.226113] acpiphp: Slot [16] registered [ 0.227102] acpiphp: Slot [17] registered [ 0.228108] acpiphp: Slot [18] registered [ 0.229129] acpiphp: Slot [19] registered [ 0.230074] acpiphp: Slot [20] registered [ 0.231127] acpiphp: Slot [21] registered [ 0.232128] acpiphp: Slot [22] registered [ 0.233104] acpiphp: Slot [23] registered [ 0.234099] acpiphp: Slot [24] registered [ 0.235112] acpiphp: Slot [25] registered [ 0.236149] acpiphp: Slot [26] registered [ 0.237105] acpiphp: Slot [27] registered [ 0.238118] acpiphp: Slot [28] registered [ 0.239125] acpiphp: Slot [29] registered [ 0.240117] acpiphp: Slot [30] registered [ 0.241093] acpiphp: Slot [31] registered [ 0.242064] PCI host bridge to bus 0000:00 [ 0.243018] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window] [ 0.244022] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window] [ 0.245025] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window] [ 0.246030] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window] [ 0.247026] pci_bus 0000:00: root bus resource [mem 0x180000000-0x1ffffffff window] [ 0.248029] pci_bus 0000:00: root bus resource [bus 00-ff] [ 0.249173] pci 0000:00:00.0: [8086:1237] type 00 class 0x060000 [ 0.252488] pci 0000:00:01.0: [8086:7000] type 00 class 0x060100 [ 0.255572] pci 0000:00:01.1: [8086:7010] type 00 class 0x010180 [ 0.261018] pci 0000:00:01.1: reg 0x20: [io 0xc320-0xc32f] [ 0.264061] pci 0000:00:01.1: legacy IDE quirk: reg 0x10: [io 0x01f0-0x01f7] [ 0.265022] pci 0000:00:01.1: legacy IDE quirk: reg 0x14: [io 0x03f6] [ 0.266021] pci 0000:00:01.1: legacy IDE quirk: reg 0x18: [io 0x0170-0x0177] [ 0.267024] pci 0000:00:01.1: legacy IDE quirk: reg 0x1c: [io 0x0376] [ 0.268555] pci 0000:00:01.3: [8086:7113] type 00 class 0x068000 [ 0.271821] pci 0000:00:01.3: quirk: [io 0x0600-0x063f] claimed by PIIX4 ACPI [ 0.275043] pci 0000:00:01.3: quirk: [io 0x0700-0x070f] claimed by PIIX4 SMB [ 0.277630] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 [ 0.283016] pci 0000:00:02.0: reg 0x10: [io 0xc300-0xc31f] [ 0.295012] pci 0000:00:02.0: reg 0x20: [mem 0xfebe4000-0xfebe7fff 64bit pref] [ 0.301017] pci 0000:00:02.0: reg 0x30: [mem 0xfeb80000-0xfebbffff pref] [ 0.306849] pci 0000:00:05.0: [1af4:1001] type 00 class 0x010000 [ 0.315022] pci 0000:00:05.0: reg 0x10: [io 0xc000-0xc07f] [ 0.322018] pci 0000:00:05.0: reg 0x14: [mem 0xfebc0000-0xfebc0fff] [ 0.335017] pci 0000:00:05.0: reg 0x20: [mem 0xfebe8000-0xfebebfff 64bit pref] [ 0.348164] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 [ 0.354014] pci 0000:00:06.0: reg 0x10: [io 0xc080-0xc0ff] [ 0.366027] pci 0000:00:06.0: reg 0x14: [mem 0xfebc1000-0xfebc1fff] [ 0.380018] pci 0000:00:06.0: reg 0x20: [mem 0xfebec000-0xfebeffff 64bit pref] [ 0.389529] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 [ 0.398026] pci 0000:00:07.0: reg 0x10: [io 0xc100-0xc17f] [ 0.406040] pci 0000:00:07.0: reg 0x14: [mem 0xfebc2000-0xfebc2fff] [ 0.422024] pci 0000:00:07.0: reg 0x20: [mem 0xfebf0000-0xfebf3fff 64bit pref] [ 0.431881] pci 0000:00:08.0: [1af4:1001] type 00 class 0x010000 [ 0.438016] pci 0000:00:08.0: reg 0x10: [io 0xc180-0xc1ff] [ 0.443019] pci 0000:00:08.0: reg 0x14: [mem 0xfebc3000-0xfebc3fff] [ 0.459020] pci 0000:00:08.0: reg 0x20: [mem 0xfebf4000-0xfebf7fff 64bit pref] [ 0.470312] pci 0000:00:09.0: [1af4:1001] type 00 class 0x010000 [ 0.477033] pci 0000:00:09.0: reg 0x10: [io 0xc200-0xc27f] [ 0.487026] pci 0000:00:09.0: reg 0x14: [mem 0xfebc4000-0xfebc4fff] [ 0.504028] pci 0000:00:09.0: reg 0x20: [mem 0xfebf8000-0xfebfbfff 64bit pref] [ 0.516413] pci 0000:00:0a.0: [1af4:1001] type 00 class 0x010000 [ 0.521018] pci 0000:00:0a.0: reg 0x10: [io 0xc280-0xc2ff] [ 0.527019] pci 0000:00:0a.0: reg 0x14: [mem 0xfebc5000-0xfebc5fff] [ 0.538016] pci 0000:00:0a.0: reg 0x20: [mem 0xfebfc000-0xfebfffff 64bit pref] [ 0.548825] ACPI: PCI: Interrupt link LNKA configured for IRQ 10 [ 0.551473] ACPI: PCI: Interrupt link LNKB configured for IRQ 10 [ 0.554471] ACPI: PCI: Interrupt link LNKC configured for IRQ 11 [ 0.557498] ACPI: PCI: Interrupt link LNKD configured for IRQ 11 [ 0.559340] ACPI: PCI: Interrupt link LNKS configured for IRQ 9 [ 0.566043] iommu: Default domain type: Passthrough [ 0.567417] SCSI subsystem initialized [ 0.568690] ACPI: bus type USB registered [ 0.569000] usbcore: registered new interface driver usbfs [ 0.569098] usbcore: registered new interface driver hub [ 0.570000] usbcore: registered new device driver usb [ 0.572130] pps_core: LinuxPPS API ver. 1 registered [ 0.574012] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti [ 0.577066] PTP clock support registered [ 0.579177] EDAC MC: Ver: 3.0.0 [ 0.581358] PCI: Using ACPI for IRQ routing [ 0.583972] NetLabel: Initializing [ 0.585011] NetLabel: domain hash size = 128 [ 0.587014] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO [ 0.590102] NetLabel: unlabeled traffic allowed by default [ 0.592364] vgaarb: loaded [ 0.594258] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0 [ 0.596017] hpet0: 3 comparators, 64-bit 100.000000 MHz counter [ 0.601382] clocksource: Switched to clocksource kvm-clock [ 0.705551] VFS: Disk quotas dquot_6.6.0 [ 0.707238] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) [ 0.710025] *** VALIDATE ramfs *** [ 0.711295] *** VALIDATE hugetlbfs *** [ 0.713145] pnp: PnP ACPI init [ 0.715458] pnp: PnP ACPI: found 6 devices [ 0.732049] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns [ 0.735173] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window] [ 0.737363] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window] [ 0.739699] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window] [ 0.742364] pci_bus 0000:00: resource 7 [mem 0xc0000000-0xfebfffff window] [ 0.744402] pci_bus 0000:00: resource 8 [mem 0x180000000-0x1ffffffff window] [ 0.746613] NET: Registered protocol family 2 [ 0.749141] IP idents hash table entries: 131072 (order: 8, 1048576 bytes, vmalloc) [ 0.754062] tcp_listen_portaddr_hash hash table entries: 4096 (order: 5, 163840 bytes, vmalloc) [ 0.757822] TCP established hash table entries: 65536 (order: 7, 524288 bytes, vmalloc) [ 0.763331] TCP bind hash table entries: 65536 (order: 9, 2097152 bytes, vmalloc) [ 0.766381] TCP: Hash tables configured (established 65536 bind 65536) [ 0.768972] MPTCP token hash table entries: 8192 (order: 6, 393216 bytes, vmalloc) [ 0.772370] UDP hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 0.775632] UDP-Lite hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 0.778599] NET: Registered protocol family 1 [ 0.781906] RPC: Registered named UNIX socket transport module. [ 0.786405] RPC: Registered udp transport module. [ 0.789060] RPC: Registered tcp transport module. [ 0.791679] RPC: Registered tcp NFSv4.1 backchannel transport module. [ 0.794482] NET: Registered protocol family 44 [ 0.796085] pci 0000:00:00.0: Limiting direct PCI/PCI transfers [ 0.798335] pci 0000:00:01.0: PIIX3: Enabling Passive Release [ 0.800577] pci 0000:00:01.0: Activating ISA DMA hang workarounds [ 0.802694] PCI: CLS 0 bytes, default 64 [ 0.804238] Unpacking initramfs... [ 2.231447] debug: unmapping init [mem 0xffff8a87fcc54000-0xffff8a87fffbffff] [ 2.235547] PCI-DMA: Using software bounce buffering for IO (SWIOTLB) [ 2.237870] software IO TLB: mapped [mem 0x00000000a8000000-0x00000000ac000000] (64MB) [ 2.240566] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 2.749791] Initialise system trusted keyrings [ 2.751121] Key type blacklist registered [ 2.752708] workingset: timestamp_bits=36 max_order=20 bucket_order=0 [ 2.761622] zbud: loaded [ 2.764588] *** VALIDATE nfs *** [ 2.765995] *** VALIDATE nfs4 *** [ 2.767755] pstore: using deflate compression [ 2.771210] Platform Keyring initialized [ 2.895380] NET: Registered protocol family 38 [ 2.897177] Key type asymmetric registered [ 2.898698] Asymmetric key parser 'x509' registered [ 2.900135] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 247) [ 2.904265] io scheduler mq-deadline registered [ 2.905562] io scheduler kyber registered [ 2.906856] io scheduler bfq registered [ 2.910612] atomic64_test: passed for x86-64 platform with CX8 and with SSE [ 2.914071] shpchp: Standard Hot Plug PCI Controller Driver version: 0.4 [ 2.917062] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0 [ 2.919512] ACPI: Power Button [PWRF] [ 3.015532] ACPI: \_SB_.LNKB: Enabled at IRQ 10 [ 3.107207] ACPI: \_SB_.LNKA: Enabled at IRQ 11 [ 3.296952] ACPI: \_SB_.LNKC: Enabled at IRQ 11 [ 3.395784] ACPI: \_SB_.LNKD: Enabled at IRQ 10 [ 3.593753] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled [ 3.623225] 00:03: ttyS1 at I/O 0x2f8 (irq = 3, base_baud = 115200) is a 16550A [ 3.656403] 00:04: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A [ 3.661736] Non-volatile memory driver v1.3 [ 3.663581] Linux agpgart interface v0.103 [ 3.703179] virtio_blk virtio1: [vda] 133624 512-byte logical blocks (68.4 MB/65.2 MiB) [ 3.706467] vda: detected capacity change from 0 to 68415488 [ 3.722341] virtio_blk virtio2: [vdb] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) [ 3.725185] vdb: detected capacity change from 0 to 1073741824 [ 3.743549] virtio_blk virtio3: [vdc] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 3.746629] vdc: detected capacity change from 0 to 2621440000 [ 3.760631] virtio_blk virtio4: [vdd] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 3.763634] vdd: detected capacity change from 0 to 2621440000 [ 3.801658] virtio_blk virtio5: [vde] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 3.804550] vde: detected capacity change from 0 to 4294967296 [ 3.823065] virtio_blk virtio6: [vdf] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 3.826151] vdf: detected capacity change from 0 to 4294967296 [ 3.838552] libphy: Fixed MDIO Bus: probed [ 3.845629] usbcore: registered new interface driver usbserial_generic [ 3.848212] usbserial: USB Serial support registered for generic [ 3.850800] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12 [ 3.855068] serio: i8042 KBD port at 0x60,0x64 irq 1 [ 3.856990] serio: i8042 AUX port at 0x60,0x64 irq 12 [ 3.859699] mousedev: PS/2 mouse device common for all mice [ 3.862536] rtc_cmos 00:05: RTC can wake from S4 [ 3.865554] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input1 [ 3.866522] rtc_cmos 00:05: registered as rtc0 [ 3.871804] rtc_cmos 00:05: alarms up to one day, y3k, 242 bytes nvram, hpet irqs [ 3.872937] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input4 [ 3.876145] intel_pstate: CPU model not supported [ 3.884649] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input3 [ 3.888420] hid: raw HID events driver (C) Jiri Kosina [ 3.890780] usbcore: registered new interface driver usbhid [ 3.892740] usbhid: USB HID core driver [ 3.893935] drop_monitor: Initializing network drop monitor service [ 3.896557] Initializing XFRM netlink socket [ 3.898597] NET: Registered protocol family 10 [ 3.902734] Segment Routing with IPv6 [ 3.904351] NET: Registered protocol family 17 [ 3.907142] mpls_gso: MPLS GSO support [ 3.917508] RAS: Correctable Errors collector initialized. [ 3.919702] AVX version of gcm_enc/dec engaged. [ 3.921104] AES CTR mode by8 optimization enabled [ 3.994034] sched_clock: Marking stable (3994007505, 0)->(5047154393, -1053146888) [ 3.998642] registered taskstats version 1 [ 4.000271] Loading compiled-in X.509 certificates [ 4.002382] zswap: loaded using pool lzo/zbud [ 4.031845] Key type big_key registered [ 4.043643] Key type encrypted registered [ 4.045649] ima: No TPM chip found, activating TPM-bypass! [ 4.047761] ima: Allocated hash algorithm: sha1 [ 4.049971] ima: No architecture policies found [ 4.051738] evm: Initialising EVM extended attributes: [ 4.053559] evm: security.selinux [ 4.054621] evm: security.ima [ 4.055608] evm: security.capability [ 4.056788] evm: HMAC attrs: 0x1 [ 4.059615] rtc_cmos 00:05: setting system clock to 2025-11-14 00:38:11 UTC (1763080691) [ 4.065686] debug: unmapping init [mem 0xffffffffa5603000-0xffffffffa57fffff] [ 4.068512] debug: unmapping init [mem 0xffffffffa4382000-0xffffffffa4658fff] [ 4.077086] Write protecting the kernel read-only data: 28672k [ 4.080078] debug: unmapping init [mem 0xffffffffa2a03000-0xffffffffa2bfffff] [ 4.083798] debug: unmapping init [mem 0xffffffffa3314000-0xffffffffa33fffff] [ 4.116832] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 4.124228] systemd[1]: Detected virtualization kvm. [ 4.125958] systemd[1]: Detected architecture x86-64. [ 4.127694] systemd[1]: Running in initial RAM disk. Welcome to Rocky Linux 8.10 (Green Obsidian) dracut-049-233.git20240115.el8 (Initramfs)! [ 4.157598] systemd[1]: No hostname configured. [ 4.159355] systemd[1]: Set hostname to . [ 4.162513] random: systemd: uninitialized urandom read (16 bytes read) [ 4.165813] systemd[1]: Initializing machine ID from random generator. [ 4.202850] random: ln: uninitialized urandom read (6 bytes read) [ 4.335718] random: systemd: uninitialized urandom read (16 bytes read) [ 4.338615] systemd[1]: Listening on Journal Socket. [ OK ] Listening on Journal Socket. [ 4.343191] systemd[1]: Reached target Timers. [ OK ] Reached target Timers. [ 4.347117] systemd[1]: Listening on Journal Socket (/dev/log). [ OK ] Listening on Journal Socket (/dev/log). Starting Journal Service... [ OK ] Reached target Swap. [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Listening on udev Control Socket. Starting Setup Virtual Console... [ OK ] Reached target Local Encrypted Volumes. [ OK ] Reached target Paths. [ OK ] Listening on udev Kernel Socket. [ OK ] Reached target Sockets. [ OK ] Reached target Initrd Root Device. [ OK ] Reached target Local File Systems. Starting Create Volatile Files and Directories... Starting Apply Kernel Variables... Starting Create list of required st…ce nodes for the current kernel... [ OK ] Started Memstrack Anylazing Service. [ OK ] Reached target Slices. [ OK ] Started Setup Virtual Console. [ OK ] Started Create Volatile Files and Directories. [ OK ] Started Apply Kernel Variables. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Started Journal Service. Starting Create Static Device Nodes in /dev... Starting dracut cmdline hook... [ OK ] Started Create Static Device Nodes in /dev. [ OK ] Started dracut cmdline hook. Starting dracut pre-udev hook... [ 5.106429] device-mapper: uevent: version 1.0.3 [ 5.108889] device-mapper: ioctl: 4.46.0-ioctl (2022-02-22) initialised: dm-devel@redhat.com [ OK ] Started dracut pre-udev hook. Starting udev Kernel Device Manager... [ OK ] Started udev Kernel Device Manager. Starting dracut pre-trigger hook... [ OK ] Started dracut pre-trigger hook. Starting udev Coldplug all Devices... Mounting Kernel Configuration File System... [ OK ] Mounted Kernel Configuration File System. [ OK ] Started udev Coldplug all Devices. [ OK ] Reached target System Initialization. [ OK ] Reached target Basic System. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. Starting dracut initqueue hook... [ 5.881834] virtio_net virtio0 ens2: renamed from eth0 [ 5.945145] random: fast init done [ 5.956537] scsi host0: ata_piix [ 5.997326] scsi host1: ata_piix [ 5.999085] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc320 irq 14 [ 6.001851] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc328 irq 15 [ 9.723276] dracut-initqueue[602]: RTNETLINK answers: File exists [ 10.563589] random: crng init done [ 10.565073] random: 7 urandom warning(s) missed due to ratelimiting Starting nbd nbd0... [ OK ] Started nbd nbd0. [ OK ] Started dracut initqueue hook. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Mounting /sysroot... [ 11.140598] EXT4-fs (nbd0): mounted filesystem with ordered data mode. Opts: (null) [ OK ] Mounted /sysroot. [ OK ] Reached target Initrd Root File System. Starting Reload Configuration from the Real Root... [ OK ] Started Reload Configuration from the Real Root. [ OK ] Reached target Initrd File Systems. [ OK ] Reached target Initrd Default Target. Starting dracut pre-pivot and cleanup hook... [ OK ] Started dracut pre-pivot and cleanup hook. Starting Cleaning Up and Shutting Down Daemons... [ OK ] Stopped target Timers. Stopping Hardware RNG Entropy Gatherer Daemon... [ OK ] Stopped dracut pre-pivot and cleanup hook. [ OK ] Stopped target Remote File Systems. [ OK ] Stopped target Remote File Systems (Pre). [ OK ] Stopped dracut initqueue hook. [ OK ] Stopped target Initrd Default Target. [ OK ] Stopped target Initrd Root Device. [ OK ] Stopped Hardware RNG Entropy Gatherer Daemon. [ OK ] Started Cleaning Up and Shutting Down Daemons. [ OK ] Stopped target Basic System. [ OK ] Stopped target Sockets. [ OK ] Stopped target Paths. [ OK ] Stopped target Slices. [ OK ] Stopped target System Initialization. [ OK ] Stopped target Local Encrypted Volumes. [ OK ] Stopped Dispatch Password Requests to Console Directory Watch. [ OK ] Stopped udev Coldplug all Devices. [ OK ] Stopped dracut pre-trigger hook. Stopping udev Kernel Device Manager... [ OK ] Stopped Create Volatile Files and Directories. [ OK ] Stopped Apply Kernel Variables. [ OK ] Stopped target Local File Systems. [ OK ] Stopped target Swap. [ OK ] Stopped udev Kernel Device Manager. [ OK ] Stopped dracut pre-udev hook. [ OK ] Stopped dracut cmdline hook. [ OK ] Stopped Create Static Device Nodes in /dev. [ OK ] Stopped Create list of required sta…vice nodes for the current kernel. [ OK ] Closed udev Control Socket. [ OK ] Closed udev Kernel Socket. Starting Cleanup udevd DB... [ OK ] Started Cleanup udevd DB. [ OK ] Reached target Switch Root. Starting Switch Root... [ 12.444451] printk: systemd: 22 output lines suppressed due to ratelimiting [ 12.725237] SELinux: Disabled at runtime. [ 12.790737] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 12.800580] systemd[1]: Detected virtualization kvm. [ 12.802688] systemd[1]: Detected architecture x86-64. Welcome to Rocky Linux 8.10 (Green Obsidian)! [ 13.335471] systemd[1]: initrd-switch-root.service: Succeeded. [ 13.338994] systemd[1]: Stopped Switch Root. [ OK ] Stopped Switch Root. [ 13.343606] systemd[1]: systemd-journald.service: Service has no hold-off time (RestartSec=0), scheduling restart. [ 13.348056] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. [ 13.352691] systemd[1]: Stopped Journal Service. [ OK ] Stopped Journal Service. [ 13.360422] systemd[1]: Starting Journal Service... Starting Journal Service... [ 13.378081] systemd[1]: Listening on Process Core Dump Socket. [ OK ] Listening on Process Core Dump Socket. [ OK ] Listening on udev Kernel Socket. [ OK ] Created slice system-getty.slice. [ OK ] Created slice system-sshd\x2dkeygen.slice. [ OK ] Listening on RPCbind Server Activation Socket. Mounting Kernel Debug File System... Activating swap /dev/disk/by-label/SWAP... Mounting POSIX Message Queue File System... [ OK ] Started Forward Password Requests to Wall Directory Watch. Mounting Huge Pages File System... [ OK ] Created slice User and Session Slice. [ OK ] Listening on udev Control Socket. Starting udev Coldplug all Devices... [ OK ] Reached target Slices. [ 13.462251] Adding 1048572k swap on /dev/vdb. Priority:-2 extents:1 across:1048572k FS [FAILED] Failed to set up automount Arbitrar…rmats File System Automount Point. See 'systemctl status proc-sys-fs-binfmt_misc.automount' for details. Starting Create list of required st…ce nodes for the current kernel... [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Reached target Paths. [ OK ] Reached target Local Encrypted Volumes. Starting Apply Kernel Variables... [ OK ] Stopped target Switch Root. [ OK ] Stopped target Initrd Root File System. [ OK ] Stopped target Initrd File Systems. Starting Remount Root and Kernel File Systems... [ OK ] Reached target RPC Port Mapper. [ OK ] Reached target rpc_pipefs.target. [ OK ] Listening on initctl Compatibility Named Pipe. [ OK ] Created slice system-serial\x2dgetty.slice. [ OK ] Started Journal Service. [ OK ] Mounted Kernel Debug File System. [ OK ] Activated swap /dev/disk/by-label/SWAP. [ OK ] Mounted POSIX Message Queue File System. [ OK ] Mounted Huge Pages File System. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Started Apply Kernel Variables. [FAILED] Failed to start Remount Root and Kernel File Systems. See 'systemctl status systemd-remount-fs.service' for details. Starting Configure read-only root support... Starting Create Static Device Nodes in /dev... [ OK ] Reached target Swap. Starting Flush Journal to Persistent Storage... [ OK ] Started Flush Journal to Persistent Storage. [ OK ] Started Create Static Device Nodes in /dev. [ OK ] Started udev Coldplug all Devices. Starting udev Kernel Device Manager... [ OK ] Reached target Local File Systems (Pre). Mounting /mnt... Mounting /home/green/git/lustre-release... [ OK ] Mounted /mnt. [ 13.863444] squashfs: version 4.0 (2009/01/31) Phillip Lougher [ OK ] Mounted /home/green/git/lustre-release. [ OK ] Started udev Kernel Device Manager. [ 14.119838] piix4_smbus 0000:00:01.3: SMBus Host Controller at 0x700, revision 0 [ 14.192709] input: PC Speaker as /devices/platform/pcspkr/input/input5 [ 14.327057] RAPL PMU: API unit is 2^-32 Joules, 0 fixed counters, 10737418240 ms ovfl timer [ 14.350635] EDAC sbridge: Ver: 1.1.2 [ 16.277491] Key type dns_resolver registered [ 16.611113] NFS: Registering the id_resolver key type [ 16.613284] Key type id_resolver registered [ 16.614775] Key type id_legacy registered [ OK ] Started Configure read-only root support. Starting Load/Save Random Seed... [ OK ] Reached target Local File Systems. Starting Create Volatile Files and Directories... Starting Rebuild Dynamic Linker Cache... Starting Mark the need to relabel after reboot... [ OK ] Started Load/Save Random Seed. [ OK ] Started Mark the need to relabel after reboot. [ OK ] Started Create Volatile Files and Directories. Starting RPC Bind... Starting Update UTMP about System Boot/Shutdown... [ OK ] Started Update UTMP about System Boot/Shutdown. [ OK ] Started RPC Bind. [ OK ] Started Rebuild Dynamic Linker Cache. Starting Update is Completed... [ OK ] Started Update is Completed. [ OK ] Reached target System Initialization. [ OK ] Listening on D-Bus System Message Bus Socket. [ OK ] Reached target Sockets. [ OK ] Reached target Basic System. Starting Login Service... Starting Restore /run/initramfs on shutdown... [ OK ] Started dnf makecache --timer. [ OK ] Started Daily Cleanup of Temporary Directories. [ OK ] Started irqbalance daemon. [ OK ] Started daily update of the root trust anchor for DNSSEC. [ OK ] Reached target Timers. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. [ OK ] Started D-Bus System Message Bus. Starting Network Manager... [ OK ] Reached target sshd-keygen.target. [ OK ] Started Restore /run/initramfs on shutdown. [ OK ] Started Login Service. [ OK ] Started Network Manager. [ OK ] Reached target Network. Starting OpenSSH server daemon... Starting Dynamic System Tuning Daemon... Starting GSSAPI Proxy Daemon... Starting Network Manager Wait Online... Starting Hostname Service... [ OK ] Started OpenSSH server daemon. [ OK ] Started GSSAPI Proxy Daemon. [ OK ] Reached target NFS client services. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Starting Permit User Sessions... [ OK ] Started Permit User Sessions. [ OK ] Started Getty on tty1. [ OK ] Started Serial Getty on ttyS0. [ OK ] Started Command Scheduler. [ OK ] Started Serial Getty on ttyS1. [ OK ] Reached target Login Prompts. [ OK ] Started Hostname Service. Starting Network Manager Script Dispatcher Service... [ OK ] Started Network Manager Script Dispatcher Service. [ OK ] Started Network Manager Wait Online. [ OK ] Reached target Network is Online. Starting Crash recovery kernel arming... Starting Notify NFS peers of a restart... Starting System Logging Service... [ OK ] Started Notify NFS peers of a restart. [ OK ] Started System Logging Service. Starting Authorization Manager... [ OK ] Started Authorization Manager. [ OK ] Started Dynamic System Tuning Daemon. [ OK ] Reached target Multi-User System. [ OK ] Reached target Graphical Interface. Starting Update UTMP about System Runlevel Changes... [ OK ] Started Update UTMP about System Runlevel Changes. Rocky Linux 8.10 (Green Obsidian) Kernel 4.18.0rh8.10-debug on an x86_64 oleg454-server login: [ 45.203494] libcfs: loading out-of-tree module taints kernel. [ 45.221143] Key type ._llcrypt registered [ 45.222914] Key type .llcrypt registered [ 45.273678] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing set_hostid [ 62.283891] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing load_modules_local [ 64.771319] hrtimer: interrupt took 6304885 ns [ 64.811153] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 64.840018] alg: No test for adler32 (adler32-zlib) [ 66.590598] Lustre: Lustre: Build Version: 2.16.61_1_gaa570bc [ 68.073074] LNet: Added LNI 192.168.204.154@tcp [8/256/0/180] [ 69.967342] Key type lgssc registered [ 72.491650] Lustre: Echo OBD driver; http://www.lustre.org/ [ 91.371404] ZFS: Loaded module v2.3.2-1, ZFS pool version 5000, ZFS filesystem version 5 [ 92.498641] LDISKFS-fs (vdc): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 100.719536] LDISKFS-fs (vdd): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 107.375792] LDISKFS-fs (vde): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 114.438697] LDISKFS-fs (vdf): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 129.011406] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing load_modules_local [ 141.123081] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 141.199758] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 141.216239] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 142.669400] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 142.725678] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 142.796569] Lustre: lustre-MDT0000: new disk, initializing [ 142.976482] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 143.006653] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 146.633677] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 158.933501] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 159.038968] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 159.129814] Lustre: 6522:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 159.184094] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 159.186947] Lustre: Skipped 1 previous similar message [ 159.244867] Lustre: lustre-MDT0001: new disk, initializing [ 159.329857] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 159.373444] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 159.384040] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 162.681731] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 167.107796] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 176.472668] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 176.551970] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 176.693418] Lustre: lustre-OST0000: new disk, initializing [ 176.695897] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 176.737681] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 176.921934] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 176.950114] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 177.040823] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 181.545715] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 194.944328] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 195.009534] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 195.128734] Lustre: lustre-OST0001: new disk, initializing [ 195.131807] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 195.221351] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 200.605685] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 203.839397] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 203.867901] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 203.945726] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 213.232622] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 223.238067] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 233.300931] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing check_logdir /tmp/testlogs/ [ 237.117561] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing yml_node [ 240.442560] Lustre: DEBUG MARKER: Client: 2.16.61.1 [ 243.000713] Lustre: DEBUG MARKER: MDS: 2.16.61.1 [ 245.538129] Lustre: DEBUG MARKER: OSS: 2.16.61.1 [ 247.160589] Lustre: DEBUG MARKER: -----============= acceptance-small: conf-sanity ============----- Thu Nov 13 19:42:12 EST 2025 [ 264.012868] Lustre: DEBUG MARKER: excepting tests: 102 106 115 32newtarball [ 265.578317] Lustre: DEBUG MARKER: skipping tests SLOW=no: 45 69 106 111 114 [ 282.593831] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 282.598522] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 282.612831] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 286.182837] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 286.182851] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 286.207709] Lustre: Skipped 2 previous similar messages [ 287.476249] LustreError: 12230:0:(obd_class.h:479:obd_check_dev()) Device 28 not setup [ 287.598182] Lustre: server umount lustre-MDT0000 complete [ 291.785245] LustreError: 6517:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763080979 with bad export cookie 10586842795491964091 [ 291.797461] LustreError: MGC192.168.204.154@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 291.804486] LustreError: 6517:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 1 previous similar message [ 296.424187] LustreError: 6532:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 296.424544] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 296.426714] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 296.426725] Lustre: Skipped 3 previous similar messages [ 296.434080] LustreError: 6532:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 6 previous similar messages [ 296.461966] Lustre: Skipped 1 previous similar message [ 301.537762] LustreError: 6532:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 301.540261] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 301.569871] LustreError: 6532:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 301.591282] Lustre: Skipped 2 previous similar messages [ 302.561543] LustreError: 6531:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 302.584682] LustreError: 6531:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 306.143783] Lustre: lustre-MDT0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 306.289451] LustreError: 12431:0:(obd_class.h:479:obd_check_dev()) Device 27 not setup [ 306.302375] LustreError: 12431:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 306.399970] Lustre: server umount lustre-MDT0001 complete [ 310.211156] LustreError: 12635:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 310.220713] LustreError: 12635:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 310.344392] Lustre: server umount lustre-OST0000 complete [ 314.801105] LustreError: 12835:0:(obd_class.h:479:obd_check_dev()) Device 2 not setup [ 314.806698] LustreError: 12835:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 315.072391] Lustre: server umount lustre-OST0001 complete [ 321.942834] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing set_hostid [ 330.925642] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing load_modules_local [ 341.053528] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 347.769135] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 354.157369] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 360.118152] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 366.962809] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 367.020543] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 367.232748] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 367.280388] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 367.356886] Lustre: lustre-MDT0000: new disk, initializing [ 367.441956] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 367.463214] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 371.252237] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing set_default_debug -1 all [ 381.369301] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 381.471523] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 381.612083] Lustre: 15024:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 381.715190] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 381.726438] Lustre: Skipped 1 previous similar message [ 381.806905] Lustre: lustre-MDT0001: new disk, initializing [ 381.916045] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 381.960318] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 381.976086] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 385.949356] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing set_default_debug -1 all [ 393.698037] Lustre: DEBUG MARKER: oleg454-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 397.202442] Lustre: DEBUG MARKER: oleg454-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 405.320027] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 405.438125] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 405.924421] Lustre: lustre-OST0000: new disk, initializing [ 405.926958] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 406.079370] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 407.697986] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 407.702525] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 407.816809] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 412.473965] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing set_default_debug -1 all [ 420.129511] Lustre: DEBUG MARKER: oleg454-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 424.798306] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 425.032941] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 430.091745] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 430.439756] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 433.639345] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 433.639912] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 433.663783] LustreError: Skipped 1 previous similar message [ 433.671975] Lustre: Skipped 1 previous similar message [ 433.671992] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 433.689525] Lustre: Skipped 2 previous similar messages [ 439.416309] LustreError: 17965:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 439.419751] LustreError: 17965:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 439.534226] Lustre: server umount lustre-OST0000 complete [ 448.488086] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 448.498894] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 448.506929] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 448.511341] Lustre: Skipped 3 previous similar messages [ 450.506680] LustreError: 18166:0:(obd_class.h:479:obd_check_dev()) Device 22 not setup [ 450.512453] LustreError: 18166:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 450.612929] Lustre: server umount lustre-MDT0000 complete [ 454.119782] LustreError: 15035:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 454.145256] LustreError: 15035:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 455.254836] LustreError: 16771:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763081142 with bad export cookie 10586842795491965624 [ 455.256120] LustreError: MGC192.168.204.154@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 455.266299] LustreError: 16771:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 455.897568] Lustre: server umount lustre-MDT0001 complete [ 461.376483] Lustre: DEBUG MARKER: == conf-sanity test 76a: set permanent params with lctl across mounts ========================================================== 19:45:47 (1763081147) [ 468.014776] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 468.708685] LustreError: 18893:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 468.789391] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 472.811691] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing set_default_debug -1 all [ 479.205419] LustreError: 18893:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 479.232079] LustreError: 18893:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 481.583910] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 486.268863] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing set_default_debug -1 all [ 491.072846] Lustre: DEBUG MARKER: oleg454-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 494.565822] Lustre: DEBUG MARKER: oleg454-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 501.374891] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 506.903195] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing set_default_debug -1 all [ 512.372349] Lustre: DEBUG MARKER: oleg454-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 521.126885] Lustre: Modifying parameter general.osc.*.max_dirty_mb=452 in log params [ 528.355722] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 528.379058] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 528.392271] Lustre: Skipped 1 previous similar message [ 528.401157] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 528.404885] Lustre: Skipped 1 previous similar message [ 533.579867] LustreError: 21196:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup [ 533.587688] LustreError: 21196:0:(obd_class.h:479:obd_check_dev()) Skipped 13 previous similar messages [ 533.683757] Lustre: server umount lustre-MDT0000 complete [ 535.543330] LustreError: 18893:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 535.563483] LustreError: 18893:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 537.209394] LustreError: 19612:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763081224 with bad export cookie 10586842795491966506 [ 537.217994] LustreError: 19612:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 1 previous similar message [ 537.220145] LustreError: MGC192.168.204.154@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 537.516566] Lustre: server umount lustre-MDT0001 complete [ 550.753082] Lustre: server umount lustre-OST0000 complete [ 569.223495] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing load_modules_local [ 578.538314] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 578.851618] LustreError: 22925:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 578.913478] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 578.916642] Lustre: Skipped 2 previous similar messages [ 582.066981] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing set_default_debug -1 all [ 589.651749] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 594.775772] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing set_default_debug -1 all [ 597.812080] Lustre: Setting parameter general.debug_raw_pointers=Y in log params [ 604.015075] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 609.123739] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing set_default_debug -1 all [ 616.547253] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 616.624515] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 616.765936] Lustre: lustre-OST0001: new disk, initializing [ 616.769939] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 618.716616] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 618.725749] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 618.829800] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 622.016901] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing set_default_debug -1 all [ 630.994331] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 634.386917] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 648.208089] LustreError: 26407:0:(obd_class.h:479:obd_check_dev()) Device 28 not setup [ 648.210311] LustreError: 26407:0:(obd_class.h:479:obd_check_dev()) Skipped 17 previous similar messages [ 648.272535] Lustre: server umount lustre-MDT0000 complete [ 649.698314] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 649.706264] LustreError: 22920:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 649.714148] Lustre: Skipped 4 previous similar messages [ 649.740813] LustreError: 22920:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 5 previous similar messages [ 651.193994] LustreError: 22905:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763081338 with bad export cookie 10586842795491967353 [ 651.197251] LustreError: MGC192.168.204.154@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 651.211382] LustreError: 22905:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 653.443149] Lustre: server umount lustre-MDT0001 complete [ 666.550347] Lustre: server umount lustre-OST0000 complete [ 680.417699] Lustre: server umount lustre-OST0001 complete [ 693.858462] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing load_modules_local [ 702.817500] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 706.923971] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing set_default_debug -1 all [ 716.137876] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 716.463676] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 716.482517] Lustre: Skipped 4 previous similar messages [ 720.964255] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing set_default_debug -1 all [ 724.296554] Lustre: 29304:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 730.329699] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 735.468241] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing set_default_debug -1 all [ 743.676721] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 748.711522] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing set_default_debug -1 all [ 755.643629] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 759.351513] Lustre: 31158:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 772.576137] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 772.586205] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 772.602579] Lustre: Skipped 1 previous similar message [ 772.605524] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 772.608255] Lustre: Skipped 3 previous similar messages [ 776.573782] LustreError: 31361:0:(obd_class.h:479:obd_check_dev()) Device 10 not setup [ 776.582741] LustreError: 31361:0:(obd_class.h:479:obd_check_dev()) Skipped 25 previous similar messages [ 776.708785] Lustre: server umount lustre-MDT0000 complete [ 779.238199] LustreError: 28187:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 779.263898] LustreError: 28187:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 7 previous similar messages [ 780.193979] LustreError: 28169:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763081467 with bad export cookie 10586842795491968648 [ 780.196550] LustreError: MGC192.168.204.154@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 780.214532] LustreError: 28169:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 780.491149] Lustre: server umount lustre-MDT0001 complete [ 793.724001] Lustre: server umount lustre-OST0000 complete [ 806.737796] Lustre: server umount lustre-OST0001 complete [ 814.253489] Lustre: DEBUG MARKER: == conf-sanity test 76b: verify params log setup correctly ========================================================== 19:51:40 (1763081500) [ 838.142933] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing load_modules_local [ 847.770775] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 851.839729] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing set_default_debug -1 all [ 860.002969] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 864.428217] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing set_default_debug -1 all [ 867.650487] Lustre: 35063:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 874.650576] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 880.120110] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing set_default_debug -1 all [ 888.425637] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 894.066125] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing set_default_debug -1 all [ 900.739995] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 904.108391] Lustre: 36921:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 913.889056] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 913.901847] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 913.909030] Lustre: Skipped 6 previous similar messages [ 913.920651] Lustre: Skipped 7 previous similar messages [ 918.950913] LustreError: 37073:0:(obd_class.h:479:obd_check_dev()) Device 10 not setup [ 918.958695] LustreError: 37073:0:(obd_class.h:479:obd_check_dev()) Skipped 25 previous similar messages [ 919.122416] Lustre: server umount lustre-MDT0000 complete [ 922.338174] LustreError: 34681:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763081609 with bad export cookie 10586842795491969754 [ 922.343207] LustreError: MGC192.168.204.154@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 922.346134] LustreError: 34681:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 922.579765] Lustre: server umount lustre-MDT0001 complete [ 936.639710] Lustre: server umount lustre-OST0000 complete [ 950.302810] Lustre: server umount lustre-OST0001 complete [ 958.264509] Lustre: DEBUG MARKER: == conf-sanity test 76c: verify changelog_mask is applied with lctl set_param -P ========================================================== 19:54:03 (1763081643) [ 973.326591] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing load_modules_local [ 984.809654] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 985.551525] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 985.564395] Lustre: Skipped 6 previous similar messages [ 990.060139] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing set_default_debug -1 all [ 999.164995] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1003.699724] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing set_default_debug -1 all [ 1006.717130] Lustre: 40164:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1013.319425] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1019.154697] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing set_default_debug -1 all [ 1026.125036] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1030.942951] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing set_default_debug -1 all [ 1037.541644] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1040.709920] Lustre: 42020:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 1046.790324] Lustre: Setting parameter general.mdd.*.changelog_mask=-CLOSE in log params [ 1046.794615] Lustre: Skipped 1 previous similar message [ 1050.592325] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1050.596419] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1050.600170] Lustre: Skipped 1 previous similar message [ 1055.994172] Lustre: server umount lustre-MDT0000 complete [ 1058.760390] LustreError: 39026:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763081746 with bad export cookie 10586842795491970860 [ 1058.762904] LustreError: MGC192.168.204.154@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1058.773495] LustreError: 39026:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 1 previous similar message [ 1058.889203] LustreError: 39040:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 192.168.204.54@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1058.896377] LustreError: 39040:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 12 previous similar messages [ 1073.120388] Lustre: lustre-MDT0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 1073.379540] Lustre: server umount lustre-MDT0001 complete [ 1080.654980] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1084.394937] LustreError: 3665:0:(client.c:1390:ptlrpc_import_delay_req()) @@@ invalidate in flight req@ffff8a8744b04700 x1848724166142080/t0(0) o250->MGC192.168.204.154@tcp@0@lo:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl Rpc:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1088.103676] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing set_default_debug -1 all [ 1095.332031] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1098.871633] Lustre: lustre-MDT0000-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 1099.389607] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing set_default_debug -1 all [ 1104.575720] Lustre: DEBUG MARKER: oleg454-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1106.181416] Lustre: DEBUG MARKER: mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec [ 1109.629836] Lustre: DEBUG MARKER: oleg454-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 1111.144590] Lustre: DEBUG MARKER: mdc.lustre-MDT0001-mdc-*.mds_server_uuid in FULL state after 0 sec [ 1116.131040] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1121.631737] Lustre: server umount lustre-MDT0000 complete [ 1124.662526] LustreError: 39028:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763081812 with bad export cookie 10586842795491972015 [ 1124.669492] LustreError: MGC192.168.204.154@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1124.673141] LustreError: 39028:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1124.930785] Lustre: server umount lustre-MDT0001 complete [ 1130.444869] Lustre: server umount lustre-OST0000 complete [ 1134.662105] Lustre: server umount lustre-OST0001 complete [ 1143.054077] Lustre: DEBUG MARKER: == conf-sanity test 76d: verify llite.*.xattr_cache can be set by 'lctl set_param -P' correctly ========================================================== 19:57:08 (1763081828) [ 1157.955737] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing load_modules_local [ 1168.661967] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1173.155808] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing set_default_debug -1 all [ 1181.997723] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1186.133751] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing set_default_debug -1 all [ 1188.566648] Lustre: 47508:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1193.886376] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1199.268640] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing set_default_debug -1 all [ 1206.153055] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1210.620188] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing set_default_debug -1 all [ 1222.330211] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1231.817431] Lustre: Setting parameter general.llite.*.xattr_cache=0 in log params [ 1244.127879] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1249.195056] LustreError: 49587:0:(obd_class.h:479:obd_check_dev()) Device 10 not setup [ 1249.200918] LustreError: 49587:0:(obd_class.h:479:obd_check_dev()) Skipped 67 previous similar messages [ 1249.281385] Lustre: server umount lustre-MDT0000 complete [ 1252.585424] LustreError: 49370:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763081940 with bad export cookie 10586842795491972820 [ 1252.592733] LustreError: MGC192.168.204.154@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1252.595318] LustreError: 49370:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 1252.921830] Lustre: server umount lustre-MDT0001 complete [ 1266.979983] Lustre: server umount lustre-OST0000 complete [ 1280.173221] Lustre: server umount lustre-OST0001 complete [ 1289.298747] Lustre: DEBUG MARKER: == conf-sanity test 77: comma-separated MGS NIDs and failover node NIDs ========================================================== 19:59:34 (1763081974) [ 1291.830181] Lustre: DEBUG MARKER: SKIP: conf-sanity test_77 mixed loopback and real device not working [ 1293.662976] Lustre: DEBUG MARKER: == conf-sanity test 78: run resize2fs on MDT and OST filesystems ========================================================== 19:59:39 (1763081979) [ 1302.988522] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing set_hostid [ 1310.068492] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing load_modules_local [ 1317.606895] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1322.124286] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1327.665346] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1327.703260] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1327.891452] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1327.947347] Lustre: lustre-MDT0000: new disk, initializing [ 1328.054259] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1331.505241] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing set_default_debug -1 all [ 1338.344211] Lustre: DEBUG MARKER: oleg454-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1343.795499] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1343.864175] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1344.062731] Lustre: lustre-OST0000: new disk, initializing [ 1344.071815] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1344.074871] Lustre: Skipped 1 previous similar message [ 1345.864174] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 1345.871647] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 1345.907057] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 1348.711042] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing set_default_debug -1 all [ 1354.435723] Lustre: DEBUG MARKER: oleg454-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1355.966150] Lustre: DEBUG MARKER: create test files [ 1376.740736] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1376.746769] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1376.752021] Lustre: Skipped 14 previous similar messages [ 1376.752866] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1376.752871] Lustre: Skipped 26 previous similar messages [ 1378.350281] Lustre: server umount lustre-OST0000 complete [ 1381.723729] Lustre: server umount lustre-MDT0000 complete [ 1387.965926] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing unload_modules_local [ 1390.225260] Key type lgssc unregistered [ 1390.475403] LNet: 54938:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1390.480989] LNetError: 54938:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1390.497704] LNet: Removed LNI 192.168.204.154@tcp [ 1391.135406] Key type .llcrypt unregistered [ 1391.140449] Key type ._llcrypt unregistered [ 1411.412451] Key type ._llcrypt registered [ 1411.414187] Key type .llcrypt registered [ 1411.537836] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing load_modules_local [ 1412.260686] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1412.494479] alg: No test for adler32 (adler32-zlib) [ 1413.525982] Lustre: Lustre: Build Version: 2.16.61_1_gaa570bc [ 1413.734072] LNet: Added LNI 192.168.204.154@tcp [8/256/0/180] [ 1415.375432] Key type lgssc registered [ 1416.417961] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1422.728401] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1422.744742] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1424.111432] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1427.115598] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing set_default_debug -1 all [ 1431.607149] Lustre: DEBUG MARKER: oleg454-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1437.488137] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1437.801248] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1442.353974] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing set_default_debug -1 all [ 1447.097580] Lustre: DEBUG MARKER: oleg454-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1449.069677] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:102 to 0x240000400:129) [ 1449.184591] Lustre: 57532:0:(osd_internal.h:1472:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 515 < left 544, rollback = 9 [ 1449.199526] Lustre: 57532:0:(osd_handler.c:2082:osd_trans_dump_creds()) create: 32/128/0, destroy: 0/0/0 [ 1449.211170] Lustre: 57532:0:(osd_handler.c:2089:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 1449.215826] Lustre: 57532:0:(osd_handler.c:2099:osd_trans_dump_creds()) write: 2/13/1, punch: 0/0/0, quota 1/3/0 [ 1449.219853] Lustre: 57532:0:(osd_handler.c:2106:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 1449.225532] Lustre: 57532:0:(osd_handler.c:2113:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 1456.999775] Lustre: DEBUG MARKER: check files after expanding the MDT and OST filesystems [ 1461.864541] Lustre: DEBUG MARKER: create more files after expanding the MDT and OST filesystems [ 1469.821775] LustreError: 58277:0:(obd_class.h:479:obd_check_dev()) Device 13 not setup [ 1469.992769] Lustre: server umount lustre-OST0000 complete [ 1482.731299] LustreError: 58478:0:(obd_class.h:479:obd_check_dev()) Device 8 not setup [ 1482.733973] LustreError: 58478:0:(obd_class.h:479:obd_check_dev()) Skipped 1 previous similar message [ 1482.918673] Lustre: server umount lustre-MDT0000 complete [ 1490.380371] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing unload_modules_local [ 1492.576771] Key type lgssc unregistered [ 1492.818111] LNet: 59011:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1492.829285] LNetError: 59011:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1492.848146] LNet: Removed LNI 192.168.204.154@tcp [ 1493.599224] Key type .llcrypt unregistered [ 1493.602378] Key type ._llcrypt unregistered [ 1517.713932] Key type ._llcrypt registered [ 1517.720735] Key type .llcrypt registered [ 1517.835941] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing load_modules_local [ 1518.894179] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1518.954719] alg: No test for adler32 (adler32-zlib) [ 1519.954410] Lustre: Lustre: Build Version: 2.16.61_1_gaa570bc [ 1520.171307] LNet: Added LNI 192.168.204.154@tcp [8/256/0/180] [ 1521.888048] Key type lgssc registered [ 1522.910805] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1529.668739] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1529.686465] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1531.091208] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1534.261844] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing set_default_debug -1 all [ 1538.636451] Lustre: DEBUG MARKER: oleg454-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1544.825986] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1545.110599] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1550.237994] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing set_default_debug -1 all [ 1555.063431] Lustre: DEBUG MARKER: oleg454-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1556.397221] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:140 to 0x240000400:161) [ 1556.502363] Lustre: 61695:0:(osd_internal.h:1472:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 515 < left 544, rollback = 9 [ 1556.506395] Lustre: 61695:0:(osd_handler.c:2082:osd_trans_dump_creds()) create: 32/128/0, destroy: 0/0/0 [ 1556.510616] Lustre: 61695:0:(osd_handler.c:2089:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 1556.515796] Lustre: 61695:0:(osd_handler.c:2099:osd_trans_dump_creds()) write: 2/13/1, punch: 0/0/0, quota 1/3/0 [ 1556.520565] Lustre: 61695:0:(osd_handler.c:2106:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 1556.524907] Lustre: 61695:0:(osd_handler.c:2113:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 1564.876575] Lustre: DEBUG MARKER: check files after shrinking the MDT and OST filesystems [ 1605.453102] LustreError: 62420:0:(obd_class.h:479:obd_check_dev()) Device 13 not setup [ 1605.565988] Lustre: server umount lustre-OST0000 complete [ 1618.927481] LustreError: 62622:0:(obd_class.h:479:obd_check_dev()) Device 8 not setup [ 1618.930569] LustreError: 62622:0:(obd_class.h:479:obd_check_dev()) Skipped 1 previous similar message [ 1619.091607] Lustre: server umount lustre-MDT0000 complete [ 1628.490721] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing unload_modules_local [ 1630.733591] Key type lgssc unregistered [ 1630.986913] LNet: 63157:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1630.997314] LNetError: 63157:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1632.041241] LNet: Removed LNI 192.168.204.154@tcp [ 1632.724268] Key type .llcrypt unregistered [ 1632.730788] Key type ._llcrypt unregistered [ 1647.804215] Key type ._llcrypt registered [ 1647.805813] Key type .llcrypt registered [ 1647.880108] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing set_hostid [ 1661.312164] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing load_modules_local [ 1662.373149] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1662.397707] alg: No test for adler32 (adler32-zlib) [ 1663.638689] Lustre: Lustre: Build Version: 2.16.61_1_gaa570bc [ 1663.933696] LNet: Added LNI 192.168.204.154@tcp [8/256/0/180] [ 1665.616588] Key type lgssc registered [ 1666.687583] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1673.710261] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1678.703216] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1682.751742] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1687.854117] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1694.081182] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1694.128567] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1694.148627] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1695.437212] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 1695.483621] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1695.553324] Lustre: lustre-MDT0000: new disk, initializing [ 1695.661551] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1695.685318] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1698.749740] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing set_default_debug -1 all [ 1707.101208] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1707.167359] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1707.271558] Lustre: 66283:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 1707.319352] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 1707.322940] Lustre: Skipped 1 previous similar message [ 1707.383552] Lustre: lustre-MDT0001: new disk, initializing [ 1707.470331] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 1707.502112] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 1707.510528] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 1710.596510] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing set_default_debug -1 all [ 1716.713411] Lustre: DEBUG MARKER: oleg454-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1719.678176] Lustre: DEBUG MARKER: oleg454-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 1724.836494] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1724.912219] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1725.118688] Lustre: lustre-OST0000: new disk, initializing [ 1725.126713] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1725.203823] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1729.509476] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing set_default_debug -1 all [ 1731.642778] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 1731.647753] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 1731.751131] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 1735.838639] Lustre: DEBUG MARKER: oleg454-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1739.168556] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 1739.317484] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 1742.437567] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 1742.631521] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 1747.430662] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1747.432410] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1747.433612] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1747.443981] Lustre: Skipped 1 previous similar message [ 1750.282816] LustreError: 69223:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 1750.386538] Lustre: server umount lustre-OST0000 complete [ 1757.664194] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1757.673663] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1757.676827] Lustre: Skipped 1 previous similar message [ 1757.699655] Lustre: Skipped 2 previous similar messages [ 1760.399532] LustreError: 69424:0:(obd_class.h:479:obd_check_dev()) Device 22 not setup [ 1760.409546] LustreError: 69424:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 1760.519842] Lustre: server umount lustre-MDT0000 complete [ 1764.077043] LustreError: 66276:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763082451 with bad export cookie 11166962875212743868 [ 1764.083586] LustreError: MGC192.168.204.154@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1764.108527] LustreError: 66276:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 1 previous similar message [ 1764.231163] LustreError: 69627:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 1764.234493] LustreError: 69627:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 1764.416250] Lustre: server umount lustre-MDT0001 complete [ 1773.086416] Lustre: DEBUG MARKER: == conf-sanity test 79: format MDT/OST without mgs option (should return errors) ========================================================== 20:07:38 (1763082458) [ 1781.488783] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing load_modules_local [ 1791.314691] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1810.291733] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing set_hostid [ 1817.803766] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing load_modules_local [ 1827.870469] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1833.642710] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1839.291119] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1845.067408] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1851.716388] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1851.793564] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1852.116351] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 1852.187130] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1852.401806] Lustre: lustre-MDT0000: new disk, initializing [ 1852.567731] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1852.599940] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1857.593114] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing set_default_debug -1 all [ 1867.658951] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1867.729781] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1867.858967] Lustre: 73865:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 1867.923519] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 1867.926786] Lustre: Skipped 1 previous similar message [ 1867.983697] Lustre: lustre-MDT0001: new disk, initializing [ 1868.127285] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 1868.179410] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 1868.189468] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 1872.146261] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing set_default_debug -1 all [ 1879.172302] Lustre: DEBUG MARKER: oleg454-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1882.462850] Lustre: DEBUG MARKER: oleg454-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 1890.016406] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1890.109183] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1890.353789] Lustre: lustre-OST0000: new disk, initializing [ 1890.361758] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1890.451170] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1896.391977] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing set_default_debug -1 all [ 1899.663734] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 1899.670695] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 1899.840580] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 1903.895137] Lustre: DEBUG MARKER: oleg454-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1907.745893] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 1907.972585] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 1911.857620] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 1912.130185] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 1915.366890] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1915.369479] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1915.378405] Lustre: Skipped 1 previous similar message [ 1915.387948] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1915.392265] LustreError: Skipped 1 previous similar message [ 1920.493326] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1920.497901] Lustre: Skipped 2 previous similar messages [ 1925.608482] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1928.160253] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 1928.255208] LustreError: 76806:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 1928.270801] LustreError: 76806:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 1928.436581] Lustre: server umount lustre-OST0000 complete [ 1934.816410] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1934.820025] LustreError: Skipped 1 previous similar message [ 1934.843805] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1934.862641] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1934.865464] Lustre: Skipped 1 previous similar message [ 1938.933740] LustreError: 77009:0:(obd_class.h:479:obd_check_dev()) Device 22 not setup [ 1938.937436] LustreError: 77009:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 1939.057194] Lustre: server umount lustre-MDT0000 complete [ 1940.963920] LustreError: 73877:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1940.974154] LustreError: 73877:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 1942.911275] LustreError: 73856:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763082630 with bad export cookie 11166962875212744750 [ 1942.917618] LustreError: MGC192.168.204.154@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1942.931034] LustreError: 73856:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1943.289134] Lustre: server umount lustre-MDT0001 complete [ 1952.534158] Lustre: DEBUG MARKER: == conf-sanity test 80: mgc import reconnect race ======== 20:10:37 (1763082637) [ 1959.368892] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1959.756780] LustreError: 77884:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1959.771780] LustreError: 77884:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 1959.834204] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1963.740600] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing set_default_debug -1 all [ 1965.026053] LustreError: 77884:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1970.147081] LustreError: 77884:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1974.587502] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1978.799963] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing set_default_debug -1 all [ 1983.659304] Lustre: DEBUG MARKER: oleg454-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1987.882649] Lustre: DEBUG MARKER: oleg454-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 1995.226994] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1995.552958] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1995.555453] Lustre: Skipped 1 previous similar message [ 2000.790960] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing set_default_debug -1 all [ 2006.484498] Lustre: DEBUG MARKER: oleg454-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2010.724785] Lustre: 80225:0:(genops.c:1791:obd_export_evict_by_uuid()) MGS: evicting 321ae2b2-dbbf-45ac-b701-35c83ff96032 at adminstrative request [ 2015.225289] LustreError: MGC192.168.204.154@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2015.256832] Lustre: *** cfs_fail_loc=906, val=2147483648*** [ 2025.453953] Lustre: MGS: Client 321ae2b2-dbbf-45ac-b701-35c83ff96032 (at 0@lo) reconnecting [ 2025.482167] Lustre: Evicted from MGS (at 0@lo) after server handle changed from 0x9af90591e507cba0 to 0x9af90591e507ce40 [ 2025.499448] Lustre: MGC192.168.204.154@tcp: Connection restored to 0@lo (at 0@lo) [ 2046.821541] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2046.932501] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2047.114253] Lustre: lustre-OST0001: new disk, initializing [ 2047.116974] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 2053.139727] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing set_default_debug -1 all [ 2053.212436] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 2053.230775] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 2053.320915] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 2060.543862] Lustre: DEBUG MARKER: oleg454-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid [ 2063.847714] Lustre: lustre-OST0001-osc-MDT0001: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2063.848511] LustreError: lustre-OST0001-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2063.898320] Lustre: Skipped 1 previous similar message [ 2063.901340] LustreError: Skipped 1 previous similar message [ 2063.903496] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 2063.936180] Lustre: Skipped 3 previous similar messages [ 2069.501175] LustreError: 81360:0:(obd_class.h:479:obd_check_dev()) Device 25 not setup [ 2069.503482] LustreError: 81360:0:(obd_class.h:479:obd_check_dev()) Skipped 13 previous similar messages [ 2069.593398] Lustre: server umount lustre-OST0001 complete [ 2074.081624] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2074.088444] LustreError: 81361:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2074.095364] Lustre: Skipped 1 previous similar message [ 2074.115634] LustreError: 81361:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 2078.916653] Lustre: server umount lustre-OST0000 complete [ 2082.787803] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2088.628114] LustreError: 81765:0:(obd_class.h:479:obd_check_dev()) Device 28 not setup [ 2088.631430] LustreError: 81765:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 2088.686333] Lustre: server umount lustre-MDT0000 complete [ 2089.450190] LustreError: 79090:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2089.461645] LustreError: 79090:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 2091.718376] LustreError: 80616:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763082779 with bad export cookie 11166962875212746304 [ 2091.722856] LustreError: MGC192.168.204.154@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2091.729976] LustreError: 80616:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2091.952727] Lustre: server umount lustre-MDT0001 complete [ 2098.736361] Lustre: DEBUG MARKER: == conf-sanity test 81: sparse OST indexing ============== 20:13:04 (1763082784) [ 2100.017398] Lustre: DEBUG MARKER: SKIP: conf-sanity test_81 needs >= 3 OSTs [ 2101.665735] Lustre: DEBUG MARKER: == conf-sanity test 82a: specify OSTs for file (succeed) or directory (succeed) ========================================================== 20:13:07 (1763082787) [ 2102.903899] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82a needs >= 3 OSTs [ 2104.389839] Lustre: DEBUG MARKER: == conf-sanity test 82b: specify OSTs for file with --pool and --ost-list options ========================================================== 20:13:10 (1763082790) [ 2105.807683] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82b needs >= 4 OSTs [ 2107.438150] Lustre: DEBUG MARKER: == conf-sanity test 83: ENOSPACE on OST doesn't cause message VFS: Busy inodes after unmount ... ========================================================== 20:13:13 (1763082793) [ 2109.079549] Lustre: DEBUG MARKER: mount the OST /dev/mapper/ost1_flakey as a ldiskfs filesystem [ 2113.836488] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2114.865184] loop: module loaded [ 2114.878617] loop0: detected capacity change from 0 to 4294967296 [ 2114.973705] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 2116.413091] Lustre: DEBUG MARKER: run llverfs in partial mode on the OST ldiskfs /mnt/lustre-ost1 [ 2119.132519] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing run_llverfs /mnt/lustre-ost1 -vpl no [ 2122.284546] Lustre: DEBUG MARKER: unmount the OST /dev/mapper/ost1_flakey [ 2130.010216] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2133.460807] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2137.275100] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2140.210975] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2145.769459] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2145.853094] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2145.891879] LustreError: 84540:0:(obd_config.c:783:class_setup()) setup lustre-OST0000-osd failed (-28) [ 2145.899058] LustreError: 84540:0:(obd_mount.c:193:lustre_start_simple()) lustre-OST0000-osd setup error -28 [ 2145.908547] LustreError: 84540:0:(tgt_mount.c:2447:server_fill_super()) Unable to start osd on /dev/mapper/ost1_flakey: -28 [ 2145.914674] LustreError: 84540:0:(super25.c:199:lustre_fill_super()) llite: Unable to mount : rc = -28 [ 2163.710396] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing set_hostid [ 2173.388389] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing load_modules_local [ 2184.158972] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2190.885209] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2197.653229] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2204.011864] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2210.743996] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2210.808337] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2211.083204] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2211.130689] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2211.256686] Lustre: lustre-MDT0000: new disk, initializing [ 2211.380602] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2211.384799] Lustre: Skipped 1 previous similar message [ 2211.412965] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2215.092992] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing set_default_debug -1 all [ 2225.543708] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2225.654729] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2225.783458] Lustre: 87382:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2226.135918] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2229.811995] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing set_default_debug -1 all [ 2237.185606] Lustre: DEBUG MARKER: oleg454-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2240.912979] Lustre: DEBUG MARKER: oleg454-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2246.986300] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2247.067321] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2253.235592] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing set_default_debug -1 all [ 2256.969554] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2257.095657] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2260.542600] Lustre: DEBUG MARKER: oleg454-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2264.703921] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2264.943810] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2269.000700] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2269.316545] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2272.744099] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2272.748053] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2272.752774] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2272.796388] Lustre: Skipped 3 previous similar messages [ 2272.809932] Lustre: Skipped 8 previous similar messages [ 2277.403161] LustreError: 90325:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 2277.414074] LustreError: 90325:0:(obd_class.h:479:obd_check_dev()) Skipped 17 previous similar messages [ 2277.543778] Lustre: server umount lustre-OST0000 complete [ 2287.929598] Lustre: server umount lustre-MDT0000 complete [ 2288.100572] LustreError: 88626:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2288.114731] LustreError: 88626:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 2291.246138] LustreError: 87376:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763082978 with bad export cookie 11166962875212746899 [ 2291.249992] LustreError: MGC192.168.204.154@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2291.252883] LustreError: 87376:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2291.508984] Lustre: server umount lustre-MDT0001 complete [ 2299.612922] Lustre: DEBUG MARKER: == conf-sanity test 84: check recovery_hard_time ========= 20:16:25 (1763082985) [ 2305.473747] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2308.942895] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing set_default_debug -1 all [ 2315.255989] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2318.826617] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing set_default_debug -1 all [ 2323.609468] Lustre: DEBUG MARKER: oleg454-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2326.692728] Lustre: DEBUG MARKER: oleg454-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2331.517195] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2335.731311] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing set_default_debug -1 all [ 2339.661949] Lustre: DEBUG MARKER: oleg454-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2342.579537] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2342.777237] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2345.842504] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2346.044616] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2351.935751] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2352.002581] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2352.099923] Lustre: lustre-OST0001: new disk, initializing [ 2352.101927] Lustre: Skipped 2 previous similar messages [ 2352.107055] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 2352.117585] Lustre: Skipped 3 previous similar messages [ 2352.196384] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 2352.200463] Lustre: Skipped 5 previous similar messages [ 2357.198473] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing set_default_debug -1 all [ 2357.361921] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 2357.386679] Lustre: Skipped 2 previous similar messages [ 2357.393294] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 2357.479639] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 2363.971522] Lustre: DEBUG MARKER: oleg454-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid [ 2367.706657] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid 50 [ 2367.879917] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2371.098276] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [ 2371.258649] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2387.106598] Lustre: DEBUG MARKER: mds1 REPLAY BARRIER on lustre-MDT0000 [ 2411.478353] Lustre: Failing over lustre-MDT0000 [ 2411.771544] LustreError: 96307:0:(obd_class.h:479:obd_check_dev()) Device 28 not setup [ 2411.781735] LustreError: 96307:0:(obd_class.h:479:obd_check_dev()) Skipped 17 previous similar messages [ 2411.873157] Lustre: server umount lustre-MDT0000 complete [ 2413.025049] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2413.036227] LustreError: Skipped 2 previous similar messages [ 2413.037930] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2413.043909] Lustre: Skipped 2 previous similar messages [ 2413.048873] LustreError: 92187:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2413.061076] LustreError: 92187:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 2430.435643] Lustre: 64498:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1763083101/real 1763083101] req@00000000827e9f74 x1848725839529600/t0(0) o400->MGC192.168.204.154@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1763083117 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 2430.456617] LustreError: MGC192.168.204.154@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2434.515498] LDISKFS-fs (dm-0): 4 truncates cleaned up [ 2434.518773] LDISKFS-fs (dm-0): recovery complete [ 2434.531965] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2439.648520] LustreError: 64495:0:(client.c:1390:ptlrpc_import_delay_req()) @@@ invalidate in flight req@000000002ff80d50 x1848725839537664/t0(0) o250->MGC192.168.204.154@tcp@0@lo:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl Rpc:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 2439.933858] Lustre: lustre-MDT0000: in recovery but waiting for the first client to connect [ 2440.780592] Lustre: lustre-MDT0000: Will be in recovery for at least 1:00, or until 3 clients reconnect [ 2440.781406] Lustre: 92875:0:(ldlm_lib.c:2067:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 2443.642498] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing set_default_debug -1 all [ 2445.296091] Lustre: 95514:0:(ldlm_lib.c:2067:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 2445.300031] Lustre: lustre-MDT0000-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 2445.301078] Lustre: 95514:0:(ldlm_lib.c:2067:extend_recovery_timer()) Skipped 1 previous similar message [ 2445.395776] LustreError: 97055:0:(ldlm_lib.c:2679:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2445.730120] LustreError: 97055:0:(ldlm_lib.c:2679:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2446.069149] LustreError: 97055:0:(ldlm_lib.c:2679:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2446.081725] LustreError: 97055:0:(ldlm_lib.c:2679:replay_request_or_update()) Skipped 1 previous similar message [ 2446.415170] LustreError: 97055:0:(ldlm_lib.c:2679:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2446.418198] LustreError: 97055:0:(ldlm_lib.c:2679:replay_request_or_update()) Skipped 1 previous similar message [ 2446.434738] Lustre: 97055:0:(ldlm_lib.c:2067:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2446.448520] Lustre: 97055:0:(ldlm_lib.c:2067:extend_recovery_timer()) Skipped 7 previous similar messages [ 2447.096118] LustreError: 97055:0:(ldlm_lib.c:2679:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2447.103878] LustreError: 97055:0:(ldlm_lib.c:2679:replay_request_or_update()) Skipped 2 previous similar messages [ 2447.431195] LustreError: 97055:0:(ldlm_lib.c:2679:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2447.443736] LustreError: 97055:0:(ldlm_lib.c:2679:replay_request_or_update()) Skipped 2 previous similar messages [ 2448.734846] Lustre: 97055:0:(ldlm_lib.c:2067:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2448.743605] Lustre: 97055:0:(ldlm_lib.c:2067:extend_recovery_timer()) Skipped 6 previous similar messages [ 2449.378670] LustreError: 97055:0:(ldlm_lib.c:2679:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2449.394387] LustreError: 97055:0:(ldlm_lib.c:2679:replay_request_or_update()) Skipped 6 previous similar messages [ 2449.727934] LustreError: 97055:0:(ldlm_lib.c:2679:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2449.743272] LustreError: 97055:0:(ldlm_lib.c:2679:replay_request_or_update()) Skipped 6 previous similar messages [ 2452.905437] Lustre: 97055:0:(ldlm_lib.c:2067:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2452.921330] Lustre: 97055:0:(ldlm_lib.c:2067:extend_recovery_timer()) Skipped 12 previous similar messages [ 2453.568154] LustreError: 97055:0:(ldlm_lib.c:2679:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2453.585041] LustreError: 97055:0:(ldlm_lib.c:2679:replay_request_or_update()) Skipped 12 previous similar messages [ 2453.903292] LustreError: 97055:0:(ldlm_lib.c:2679:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2453.920572] LustreError: 97055:0:(ldlm_lib.c:2679:replay_request_or_update()) Skipped 12 previous similar messages [ 2460.987791] Lustre: 97055:0:(ldlm_lib.c:2067:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2460.997193] Lustre: 97055:0:(ldlm_lib.c:2067:extend_recovery_timer()) Skipped 24 previous similar messages [ 2461.635200] LustreError: 97055:0:(ldlm_lib.c:2679:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2461.644307] LustreError: 97055:0:(ldlm_lib.c:2679:replay_request_or_update()) Skipped 24 previous similar messages [ 2461.975868] LustreError: 97055:0:(ldlm_lib.c:2679:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2461.979145] LustreError: 97055:0:(ldlm_lib.c:2679:replay_request_or_update()) Skipped 24 previous similar messages [ 2477.266714] Lustre: 97055:0:(ldlm_lib.c:2067:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2477.281156] Lustre: 97055:0:(ldlm_lib.c:2067:extend_recovery_timer()) Skipped 50 previous similar messages [ 2477.928282] LustreError: 97055:0:(ldlm_lib.c:2679:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2477.933736] LustreError: 97055:0:(ldlm_lib.c:2679:replay_request_or_update()) Skipped 50 previous similar messages [ 2478.255182] LustreError: 97055:0:(ldlm_lib.c:2679:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2478.263940] LustreError: 97055:0:(ldlm_lib.c:2679:replay_request_or_update()) Skipped 50 previous similar messages [ 2500.127257] Lustre: 97055:0:(ldlm_lib.c:2375:target_recovery_overseer()) lustre-MDT0000 recovery is aborted by hard timeout [ 2500.140223] Lustre: 97055:0:(ldlm_lib.c:2385:target_recovery_overseer()) recovery is aborted, evict exports in recovery [ 2500.156060] Lustre: 97055:0:(genops.c:1618:class_disconnect_stale_exports()) lustre-MDT0000: disconnect stale client 08ef4005-18e7-4714-87b2-15c03446e39a@192.168.204.54@tcp [ 2500.171528] Lustre: lustre-MDT0000: disconnecting 1 stale clients [ 2500.183949] Lustre: 97055:0:(ldlm_lib.c:1897:abort_req_replay_queue()) @@@ aborted: req@00000000fc748178 x1848725824669568/t0(8589934768) o35->08ef4005-18e7-4714-87b2-15c03446e39a@192.168.204.54@tcp:403/0 lens 392/0 e 0 to 0 dl 1763083198 ref 1 fl Complete:/604/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:0 [ 2500.207629] Lustre: lustre-MDT0000-osd: cancel update llog [0x200000400:0x1:0x0] [ 2500.222744] Lustre: lustre-MDT0000: Denying connection for new client 08ef4005-18e7-4714-87b2-15c03446e39a (at 192.168.204.54@tcp), waiting for 3 known clients (2 recovered, 0 in progress, and 1 evicted) already passed deadline 0:00 [ 2500.224974] Lustre: lustre-MDT0001-osp-MDT0000: cancel update llog [0x240000401:0x1:0x0] [ 2500.272127] Lustre: lustre-MDT0000-osp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 2500.276487] Lustre: Skipped 2 previous similar messages [ 2500.285180] Lustre: lustre-MDT0000: Recovery over after 1:00, of 3 clients 2 recovered and 1 was evicted. [ 2500.348048] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:45 to 0x2c0000401:545) [ 2500.352464] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:77 to 0x280000401:577) [ 2511.337491] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2511.355207] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2511.371393] Lustre: Skipped 3 previous similar messages [ 2511.382633] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2511.388863] Lustre: Skipped 2 previous similar messages [ 2517.440650] Lustre: server umount lustre-OST0000 complete [ 2520.033644] LustreError: 92998:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2520.055541] LustreError: 92998:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 41 previous similar messages [ 2527.544202] Lustre: server umount lustre-OST0001 complete [ 2537.632208] Lustre: server umount lustre-MDT0000 complete [ 2540.890473] LustreError: 91383:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763083228 with bad export cookie 11166962875212798111 [ 2540.894069] LustreError: MGC192.168.204.154@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2540.904878] LustreError: 91383:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2541.076203] LustreError: 98324:0:(obd_class.h:479:obd_check_dev()) Device 27 not setup [ 2541.092519] LustreError: 98324:0:(obd_class.h:479:obd_check_dev()) Skipped 23 previous similar messages [ 2541.271228] Lustre: server umount lustre-MDT0001 complete [ 2548.988770] Lustre: DEBUG MARKER: == conf-sanity test 85: osd_ost init: fail ea_fid_set ==== 20:20:34 (1763083234) [ 2555.416033] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2555.429916] Lustre: *** cfs_fail_loc=197, val=0*** [ 2570.975590] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 2576.095449] LustreError: 99015:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.204.154@tcp: failed processing log, type 4: rc = -110 [ 2581.263172] Lustre: *** cfs_fail_loc=197, val=0*** [ 2581.268444] Lustre: Skipped 65 previous similar messages [ 2605.966407] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing set_default_debug -1 all [ 2610.101173] Lustre: DEBUG MARKER: oleg454-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2611.832813] Lustre: server umount lustre-OST0000 complete [ 2618.044137] Lustre: DEBUG MARKER: == conf-sanity test 86: Replacing mkfs.lustre -G option == 20:21:44 (1763083304) [ 2637.376954] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2637.936257] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2637.943175] Lustre: Skipped 2 previous similar messages [ 2641.778594] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing set_default_debug -1 all [ 2648.365993] Lustre: DEBUG MARKER: oleg454-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2648.550136] LustreError: 100525:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2648.568947] LustreError: 100525:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 6 previous similar messages [ 2651.672581] LustreError: 100556:0:(osp_object.c:617:osp_attr_get()) lustre-MDT0001-osp-MDT0000: osp_attr_get update error [0x200000009:0x1:0x0]: rc = -5 [ 2651.683448] LustreError: 100556:0:(lod_sub_object.c:917:lod_sub_prep_llog()) lustre-MDT0000-mdtlov: can't get id from catalogs: rc = -5 [ 2651.701179] LustreError: 100556:0:(lod_dev.c:510:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 14, retries 0, failed: rc = -5 [ 2651.950250] Lustre: server umount lustre-MDT0000 complete [ 2664.696370] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2676.544049] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing set_hostid [ 2684.159599] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing load_modules_local [ 2691.839822] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2696.936638] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2702.497563] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2707.764658] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2712.786805] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2712.824733] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2712.993928] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2713.022582] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2713.080662] Lustre: lustre-MDT0000: new disk, initializing [ 2713.171538] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2716.306488] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing set_default_debug -1 all [ 2724.000855] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2724.054425] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2724.107907] Lustre: 104594:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2724.308863] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2726.968015] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing set_default_debug -1 all [ 2732.031612] Lustre: DEBUG MARKER: oleg454-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2734.385577] Lustre: DEBUG MARKER: oleg454-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2738.733573] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2738.791476] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2743.301162] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing set_default_debug -1 all [ 2746.528938] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2749.672737] Lustre: DEBUG MARKER: oleg454-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2753.593421] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2753.776370] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2756.398699] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2756.571665] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2761.697156] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2761.700562] LustreError: Skipped 4 previous similar messages [ 2761.703601] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2761.712157] Lustre: Skipped 5 previous similar messages [ 2761.717499] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2761.721707] Lustre: Skipped 9 previous similar messages [ 2764.141980] Lustre: server umount lustre-OST0000 complete [ 2773.354566] Lustre: server umount lustre-MDT0000 complete [ 2775.732969] LustreError: 104585:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763083463 with bad export cookie 11166962875212808667 [ 2775.737728] LustreError: MGC192.168.204.154@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2775.740060] LustreError: 104585:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 2776.017190] Lustre: server umount lustre-MDT0001 complete [ 2781.957559] Lustre: DEBUG MARKER: == conf-sanity test 87: check if MDT inode can hold EAs with N stripes properly ========================================================== 20:24:27 (1763083467) [ 2788.584155] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing unload_modules_local [ 2790.933129] Key type lgssc unregistered [ 2791.193441] LNet: 108664:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2791.215861] LNetError: 108664:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2791.238656] LNet: Removed LNI 192.168.204.154@tcp [ 2791.919717] Key type .llcrypt unregistered [ 2791.922350] Key type ._llcrypt unregistered [ 2805.968522] Key type ._llcrypt registered [ 2805.970917] Key type .llcrypt registered [ 2806.052042] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing set_hostid [ 2815.883266] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing load_modules_local [ 2816.482832] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2816.498033] alg: No test for adler32 (adler32-zlib) [ 2817.507513] Lustre: Lustre: Build Version: 2.16.61_1_gaa570bc [ 2817.681536] LNet: Added LNI 192.168.204.154@tcp [8/256/0/180] [ 2819.303259] Key type lgssc registered [ 2820.179766] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2826.940809] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2831.753835] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2836.351852] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2841.270459] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2845.669235] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2850.800912] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2850.845436] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 2850.860392] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2852.033517] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2852.058676] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2852.131994] Lustre: lustre-MDT0000: new disk, initializing [ 2852.206426] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2852.227940] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2854.887133] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing set_default_debug -1 all [ 2863.740858] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2863.800282] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2864.053124] Lustre: lustre-OST0000: Not available for connect from 0@lo (not set up) [ 2864.057882] Lustre: lustre-OST0000: new disk, initializing [ 2864.065959] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 2864.068922] Lustre: Skipped 1 previous similar message [ 2864.118576] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2868.861456] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing set_default_debug -1 all [ 2869.273475] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 2869.279752] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 2869.319169] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 2876.486133] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2876.525960] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2876.639938] Lustre: lustre-OST0001: new disk, initializing [ 2876.645365] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 2876.722646] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 2879.831984] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:1:ost [ 2879.843780] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:1:ost] [ 2879.885804] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x280000400 [ 2881.280031] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing set_default_debug -1 all [ 2890.210516] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2890.212673] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2890.217147] Lustre: Skipped 1 previous similar message [ 2894.059190] LustreError: 114228:0:(obd_class.h:479:obd_check_dev()) Device 17 not setup [ 2894.203968] Lustre: server umount lustre-MDT0000 complete [ 2898.943093] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: (null) [ 2903.180479] LustreError: 112781:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763083590 with bad export cookie 18055424745430664863 [ 2903.186174] LustreError: MGC192.168.204.154@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2903.220117] LustreError: 114823:0:(obd_class.h:479:obd_check_dev()) Device 13 not setup [ 2903.223263] LustreError: 114823:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 2903.253721] Lustre: server umount lustre-OST0000 complete [ 2906.041866] LustreError: 115024:0:(obd_class.h:479:obd_check_dev()) Device 2 not setup [ 2906.044795] LustreError: 115024:0:(obd_class.h:479:obd_check_dev()) Skipped 1 previous similar message [ 2906.127735] Lustre: server umount lustre-OST0001 complete [ 2911.692444] Lustre: DEBUG MARKER: == conf-sanity test 88: check the default mount options can be overridden ========================================================== 20:26:37 (1763083597) [ 2917.025426] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing load_modules_local [ 2922.073104] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2922.668637] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2925.822370] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2926.469320] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2930.789442] Lustre: DEBUG MARKER: == conf-sanity test 89: check tunefs --param and --erase-param{s} options ========================================================== 20:26:56 (1763083616) [ 2939.358271] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2940.961643] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2942.666628] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2944.257465] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2945.077728] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2946.522836] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2957.622978] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing set_hostid [ 2962.791611] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing load_modules_local [ 2968.926790] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2973.471814] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2977.620427] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2982.144986] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2986.829181] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2986.883287] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2987.025559] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2987.056313] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2987.119368] Lustre: lustre-MDT0000: new disk, initializing [ 2987.186062] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2987.206145] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2989.437735] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing set_default_debug -1 all [ 2996.755421] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2996.815960] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2996.868605] Lustre: 120345:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2996.906449] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 2996.908743] Lustre: Skipped 1 previous similar message [ 2996.967318] Lustre: lustre-MDT0001: new disk, initializing [ 2997.052357] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2997.108252] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 2997.116681] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2999.666176] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing set_default_debug -1 all [ 3005.133904] Lustre: DEBUG MARKER: oleg454-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 3007.680989] Lustre: DEBUG MARKER: oleg454-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 3012.610132] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3012.651223] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3012.822517] Lustre: lustre-OST0000: new disk, initializing [ 3012.826236] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3012.887955] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3013.096758] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 3013.105222] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 3013.218845] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3017.139377] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing set_default_debug -1 all [ 3023.259264] Lustre: DEBUG MARKER: oleg454-client.virtnet: executing wait_import_state_mount (FULL|IDLE) failover.node=192.0.2.254@tcp0,osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3026.254204] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 3026.386298] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 3029.228937] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 3029.388568] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 3033.568838] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3033.572965] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3033.582092] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3033.590963] Lustre: Skipped 1 previous similar message [ 3033.595320] Lustre: Skipped 1 previous similar message [ 3038.695158] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3038.701540] Lustre: Skipped 2 previous similar messages [ 3043.816238] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3043.828725] Lustre: Skipped 1 previous similar message [ 3045.345939] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 3045.382143] LustreError: 123279:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 3045.390621] LustreError: 123279:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 3045.524128] Lustre: server umount lustre-OST0000 complete [ 3048.929412] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3053.536451] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3053.550123] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3053.575592] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3053.582884] Lustre: Skipped 1 previous similar message [ 3054.848367] LustreError: 123483:0:(obd_class.h:479:obd_check_dev()) Device 22 not setup [ 3054.852309] LustreError: 123483:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 3054.925194] Lustre: server umount lustre-MDT0000 complete [ 3057.907709] LustreError: 120337:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763083745 with bad export cookie 18055424745430666088 [ 3057.914709] LustreError: MGC192.168.204.154@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3057.932900] LustreError: 120337:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 1 previous similar message [ 3058.308184] Lustre: server umount lustre-MDT0001 complete [ 3065.185868] Lustre: DEBUG MARKER: == conf-sanity test 90a: check max_mod_rpcs_in_flight is enforced ========================================================== 20:29:11 (1763083751) [ 3070.325398] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3070.732269] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3073.525690] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing set_default_debug -1 all [ 3076.067409] LustreError: 124366:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3076.082587] LustreError: 124366:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 3079.728981] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3083.247361] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing set_default_debug -1 all [ 3087.725205] Lustre: DEBUG MARKER: oleg454-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 3091.123283] Lustre: DEBUG MARKER: oleg454-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 3096.219659] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3096.417984] LustreError: 125964:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3101.387521] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing set_default_debug -1 all [ 3105.768840] Lustre: DEBUG MARKER: oleg454-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3109.462535] Lustre: *** cfs_fail_loc=159, val=0*** [ 3109.466748] Lustre: Skipped 1 previous similar message [ 3125.845812] Lustre: lustre-MDT0000: Client 34e5bc23-7130-496c-b454-657709ad296c (at 192.168.204.54@tcp) reconnecting [ 3126.986780] Lustre: *** cfs_fail_loc=159, val=0*** [ 3126.988875] Lustre: Skipped 3 previous similar messages [ 3143.261486] Lustre: lustre-MDT0000: Client 34e5bc23-7130-496c-b454-657709ad296c (at 192.168.204.54@tcp) reconnecting [ 3147.746109] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3147.754875] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3147.768705] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3147.780318] Lustre: Skipped 2 previous similar messages [ 3150.817620] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3152.390208] LustreError: 126851:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 3152.405644] LustreError: 126851:0:(obd_class.h:479:obd_check_dev()) Skipped 13 previous similar messages [ 3152.515191] Lustre: server umount lustre-OST0000 complete [ 3155.939066] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3155.948913] Lustre: Skipped 2 previous similar messages [ 3170.271798] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 3170.363346] LustreError: 127054:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup [ 3170.374067] LustreError: 127054:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 3170.455630] Lustre: server umount lustre-MDT0000 complete [ 3171.299923] LustreError: 124361:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3171.327130] LustreError: 124361:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 3173.504275] LustreError: 124348:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763083860 with bad export cookie 18055424745430666970 [ 3173.508909] LustreError: MGC192.168.204.154@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3173.519309] LustreError: 124348:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 3173.739091] Lustre: server umount lustre-MDT0001 complete [ 3183.154190] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing unload_modules_local [ 3185.325460] Key type lgssc unregistered [ 3185.576216] LNet: 127788:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3185.583891] LNetError: 127788:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3185.599771] LNet: Removed LNI 192.168.204.154@tcp [ 3186.461776] Key type .llcrypt unregistered [ 3186.464470] Key type ._llcrypt unregistered [ 3204.735816] Key type ._llcrypt registered [ 3204.741525] Key type .llcrypt registered [ 3204.834979] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing load_modules_local [ 3205.458770] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3205.478875] alg: No test for adler32 (adler32-zlib) [ 3206.405795] Lustre: Lustre: Build Version: 2.16.61_1_gaa570bc [ 3206.590626] LNet: Added LNI 192.168.204.154@tcp [8/256/0/180] [ 3208.241363] Key type lgssc registered [ 3208.973982] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3214.627842] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3214.640557] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3216.001331] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3218.623428] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing set_default_debug -1 all [ 3225.572800] LustreError: 129161:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3225.592798] LustreError: 129161:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 3225.604432] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3225.899805] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3229.035082] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing set_default_debug -1 all [ 3233.298234] Lustre: DEBUG MARKER: oleg454-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 3236.255588] Lustre: DEBUG MARKER: oleg454-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 3240.972501] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3241.187981] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3242.225230] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:42 to 0x280000401:65) [ 3244.955990] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing set_default_debug -1 all [ 3248.971431] Lustre: DEBUG MARKER: oleg454-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3257.723550] Lustre: *** cfs_fail_loc=159, val=0*** [ 3273.328408] Lustre: lustre-MDT0000: Client 0cb6cdba-07df-4573-b7e8-f667b76f0a6b (at 192.168.204.54@tcp) reconnecting [ 3274.850530] Lustre: *** cfs_fail_loc=159, val=0*** [ 3274.852571] Lustre: Skipped 1 previous similar message [ 3290.208242] Lustre: lustre-MDT0001: Client 0cb6cdba-07df-4573-b7e8-f667b76f0a6b (at 192.168.204.54@tcp) reconnecting [ 3291.268934] Lustre: *** cfs_fail_loc=159, val=0*** [ 3291.271909] Lustre: Skipped 2 previous similar messages [ 3307.638222] Lustre: lustre-MDT0001: Client 0cb6cdba-07df-4573-b7e8-f667b76f0a6b (at 192.168.204.54@tcp) reconnecting [ 3322.619240] Lustre: 131897:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 3324.861332] Lustre: *** cfs_fail_loc=159, val=0*** [ 3324.868964] Lustre: Skipped 4 previous similar messages [ 3340.379533] Lustre: lustre-MDT0000: Client 872d2393-fe6c-4e90-87a0-c9c4f74bea9e (at 192.168.204.54@tcp) reconnecting [ 3341.343487] Lustre: *** cfs_fail_loc=159, val=0*** [ 3341.346291] Lustre: Skipped 12 previous similar messages [ 3357.781854] Lustre: lustre-MDT0000: Client 872d2393-fe6c-4e90-87a0-c9c4f74bea9e (at 192.168.204.54@tcp) reconnecting [ 3358.551454] Lustre: 132184:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 3363.812197] LustreError: lustre-OST0000-osc-MDT0000: operation ost_destroy to node 0@lo failed: rc = -107 [ 3363.813345] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3363.815568] LustreError: Skipped 2 previous similar messages [ 3363.821652] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3364.322291] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3364.324027] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3364.327523] LustreError: Skipped 1 previous similar message [ 3364.329363] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3367.151074] LustreError: 132281:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 3367.220196] Lustre: server umount lustre-OST0000 complete [ 3369.441465] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3369.443143] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3369.454693] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3369.464522] Lustre: Skipped 2 previous similar messages [ 3374.560761] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3374.567564] Lustre: Skipped 1 previous similar message [ 3375.377358] LustreError: 132483:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup [ 3375.381410] LustreError: 132483:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 3377.447142] LustreError: 132483:0:(obd_class.h:479:obd_check_dev()) Device 10 not setup [ 3377.455932] LustreError: 132483:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 3377.502947] Lustre: server umount lustre-MDT0000 complete [ 3379.681650] LustreError: 129156:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3379.693957] LustreError: 129156:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 3379.934181] LustreError: 129142:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763084067 with bad export cookie 10156688710814488684 [ 3379.938322] LustreError: MGC192.168.204.154@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3379.942132] LustreError: 129142:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 3380.026575] LustreError: 132685:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [ 3380.032243] LustreError: 132685:0:(obd_class.h:479:obd_check_dev()) Skipped 1 previous similar message [ 3380.164438] Lustre: server umount lustre-MDT0001 complete [ 3386.690515] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing unload_modules_local [ 3388.407140] Key type lgssc unregistered [ 3388.583715] LNet: 133216:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3388.597234] LNetError: 133216:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3388.609440] LNet: Removed LNI 192.168.204.154@tcp [ 3389.060282] Key type .llcrypt unregistered [ 3389.062268] Key type ._llcrypt unregistered [ 3403.134821] Key type ._llcrypt registered [ 3403.136924] Key type .llcrypt registered [ 3403.185888] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing load_modules_local [ 3403.779140] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3403.802737] alg: No test for adler32 (adler32-zlib) [ 3404.767792] Lustre: Lustre: Build Version: 2.16.61_1_gaa570bc [ 3404.895780] LNet: Added LNI 192.168.204.154@tcp [8/256/0/180] [ 3406.519228] Key type lgssc registered [ 3407.174329] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3412.173292] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3412.187920] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3413.610491] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3416.581041] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing set_default_debug -1 all [ 3422.775988] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3422.997546] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3425.445610] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing set_default_debug -1 all [ 3429.164615] Lustre: DEBUG MARKER: oleg454-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 3431.837641] Lustre: DEBUG MARKER: oleg454-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 3435.901933] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3436.101479] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3439.230275] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:129) [ 3439.558906] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing set_default_debug -1 all [ 3442.910340] Lustre: DEBUG MARKER: oleg454-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3448.431036] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:97) [ 3450.494729] Lustre: 136839:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 3451.885317] Lustre: 136889:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 3453.032290] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 in log params [ 3453.598357] Lustre: Setting parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=8 in log params [ 3455.430559] Lustre: 137037:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify lustre/mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 (mode = 0) failed: rc = -17 [ 3455.935511] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=7 in log params [ 3456.421817] Lustre: 137135:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 3459.044716] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3459.049245] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3459.057796] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3459.061534] Lustre: Skipped 1 previous similar message [ 3459.069636] Lustre: Skipped 1 previous similar message [ 3460.664162] LustreError: 137232:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 3460.757634] Lustre: server umount lustre-OST0000 complete [ 3464.162459] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3464.162579] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3464.174682] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3464.177420] Lustre: Skipped 1 previous similar message [ 3469.284274] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3469.289037] Lustre: Skipped 1 previous similar message [ 3469.372120] LustreError: 137440:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup [ 3469.380121] LustreError: 137440:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 3469.432530] Lustre: server umount lustre-MDT0000 complete [ 3471.570697] LustreError: 136852:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763084159 with bad export cookie 15121403436849053807 [ 3471.572499] LustreError: MGC192.168.204.154@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3471.577129] LustreError: 136852:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 3471.666606] LustreError: 137641:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [ 3471.669576] LustreError: 137641:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 3471.745901] Lustre: server umount lustre-MDT0001 complete [ 3477.058074] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing unload_modules_local [ 3478.508476] Key type lgssc unregistered [ 3478.698137] LNet: 138171:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3478.702298] LNetError: 138171:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3478.716688] LNet: Removed LNI 192.168.204.154@tcp [ 3479.121460] Key type .llcrypt unregistered [ 3479.123202] Key type ._llcrypt unregistered [ 3491.766563] Key type ._llcrypt registered [ 3491.768263] Key type .llcrypt registered [ 3491.829790] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing load_modules_local [ 3492.228261] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3492.275752] alg: No test for adler32 (adler32-zlib) [ 3493.178430] Lustre: Lustre: Build Version: 2.16.61_1_gaa570bc [ 3493.318551] LNet: Added LNI 192.168.204.154@tcp [8/256/0/180] [ 3494.935207] Key type lgssc registered [ 3495.549584] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3499.416569] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3499.431419] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3500.684883] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3502.557513] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing set_default_debug -1 all [ 3507.109398] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3507.344165] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3509.953827] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing set_default_debug -1 all [ 3513.027228] Lustre: DEBUG MARKER: oleg454-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 3515.197575] Lustre: DEBUG MARKER: oleg454-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 3519.103285] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3519.365162] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3522.129984] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing set_default_debug -1 all [ 3524.979220] Lustre: DEBUG MARKER: oleg454-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3526.574450] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:161) [ 3531.699546] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:129) [ 3535.264335] Lustre: *** cfs_fail_loc=159, val=0*** [ 3551.818781] Lustre: lustre-MDT0000: Client c7a3ae37-a9d0-4b42-a3a7-463d8a836afa (at 192.168.204.54@tcp) reconnecting [ 3557.352226] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3557.353589] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3557.356133] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3557.371360] Lustre: Skipped 1 previous similar message [ 3560.558023] LustreError: 141918:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 3560.625697] Lustre: server umount lustre-OST0000 complete [ 3563.488776] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3563.496020] LustreError: Skipped 1 previous similar message [ 3563.498476] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3563.506380] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3563.509561] Lustre: Skipped 1 previous similar message [ 3567.584663] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3567.584909] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3568.779312] LustreError: 142120:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup [ 3568.781848] LustreError: 142120:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 3568.820768] Lustre: server umount lustre-MDT0000 complete [ 3570.605327] LustreError: 139522:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763084258 with bad export cookie 9734033329930345319 [ 3570.608404] LustreError: MGC192.168.204.154@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3570.610029] LustreError: 139522:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 3570.688147] LustreError: 142323:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [ 3570.691330] LustreError: 142323:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 3570.751898] Lustre: server umount lustre-MDT0001 complete [ 3575.328548] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing unload_modules_local [ 3576.627269] Key type lgssc unregistered [ 3576.784595] LNet: 142854:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3576.792128] LNetError: 142854:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3576.806627] LNet: Removed LNI 192.168.204.154@tcp [ 3577.218752] Key type .llcrypt unregistered [ 3577.221117] Key type ._llcrypt unregistered [ 3589.768700] Key type ._llcrypt registered [ 3589.771213] Key type .llcrypt registered [ 3589.835202] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing load_modules_local [ 3590.329346] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3590.386385] alg: No test for adler32 (adler32-zlib) [ 3591.317213] Lustre: Lustre: Build Version: 2.16.61_1_gaa570bc [ 3591.460585] LNet: Added LNI 192.168.204.154@tcp [8/256/0/180] [ 3593.063276] Key type lgssc registered [ 3593.791151] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3598.133167] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3598.145439] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3599.417773] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3601.438138] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing set_default_debug -1 all [ 3606.584726] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3606.831619] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3609.301767] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing set_default_debug -1 all [ 3612.712322] Lustre: DEBUG MARKER: oleg454-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 3614.641466] Lustre: DEBUG MARKER: oleg454-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 3618.311378] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3618.495743] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3621.431165] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing set_default_debug -1 all [ 3623.597087] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:170 to 0x280000401:193) [ 3624.372525] Lustre: DEBUG MARKER: oleg454-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3631.792427] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:161) [ 3650.886751] Lustre: lustre-OST0000: evicting 8869cfb8-5a26-413c-9114-09bca3e16f7b (at 192.168.204.54@tcp) by administrative request [ 3650.891501] Lustre: Skipped 1 previous similar message [ 3678.179388] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3678.181525] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3678.185039] Lustre: Skipped 1 previous similar message [ 3678.185376] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3678.188300] Lustre: Skipped 1 previous similar message [ 3682.380293] LustreError: 147033:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 3682.466273] Lustre: server umount lustre-OST0000 complete [ 3688.417290] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3688.422053] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3688.426792] Lustre: Skipped 1 previous similar message [ 3690.572757] LustreError: 147235:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup [ 3690.576197] LustreError: 147235:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 3690.626428] Lustre: server umount lustre-MDT0000 complete [ 3692.670813] LustreError: 144207:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763084380 with bad export cookie 157582701230090408 [ 3692.675194] LustreError: MGC192.168.204.154@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3692.676565] LustreError: 144207:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 3692.777458] LustreError: 147436:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [ 3692.779719] LustreError: 147436:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 3692.878186] Lustre: server umount lustre-MDT0001 complete [ 3697.848233] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing unload_modules_local [ 3699.344254] Key type lgssc unregistered [ 3699.536746] LNet: 147967:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3699.540858] LNetError: 147967:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3700.584721] LNet: Removed LNI 192.168.204.154@tcp [ 3701.038494] Key type .llcrypt unregistered [ 3701.041215] Key type ._llcrypt unregistered [ 3737.258036] Key type ._llcrypt registered [ 3737.260095] Key type .llcrypt registered [ 3737.314455] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing load_modules_local [ 3737.820178] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3737.859634] alg: No test for adler32 (adler32-zlib) [ 3738.788958] Lustre: Lustre: Build Version: 2.16.61_1_gaa570bc [ 3738.912661] LNet: Added LNI 192.168.204.154@tcp [8/256/0/180] [ 3740.527303] Key type lgssc registered [ 3741.112430] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3745.096634] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3745.107510] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3746.353283] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3748.032427] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing set_default_debug -1 all [ 3752.116929] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3752.330679] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3754.388550] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing set_default_debug -1 all [ 3757.106876] Lustre: DEBUG MARKER: oleg454-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 3758.816745] Lustre: DEBUG MARKER: oleg454-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 3762.447034] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3762.640074] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3765.503616] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing set_default_debug -1 all [ 3768.030554] Lustre: DEBUG MARKER: oleg454-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3770.853922] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:195 to 0x280000401:225) [ 3777.004874] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:193) [ 3786.210102] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3786.214427] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3786.214866] LustreError: Skipped 1 previous similar message [ 3786.221173] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3787.234412] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3787.234596] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3787.237671] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3788.527093] LustreError: 152685:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 3788.571023] Lustre: server umount lustre-OST0000 complete [ 3792.352594] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3792.353586] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3792.357336] Lustre: Skipped 1 previous similar message [ 3792.360103] Lustre: Skipped 1 previous similar message [ 3796.430510] LustreError: 152886:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup [ 3796.433204] LustreError: 152886:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 3796.477612] Lustre: server umount lustre-MDT0000 complete [ 3797.472433] LustreError: 150408:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3797.480114] LustreError: 150408:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 3798.174767] LustreError: 152887:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763084485 with bad export cookie 15483914270890172509 [ 3798.177766] LustreError: MGC192.168.204.154@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3798.181065] LustreError: 152887:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 3798.255894] LustreError: 153088:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [ 3798.258021] LustreError: 153088:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 3798.334973] Lustre: server umount lustre-MDT0001 complete [ 3802.985529] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing unload_modules_local [ 3804.364217] Key type lgssc unregistered [ 3804.550902] LNet: 153619:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3804.556231] LNetError: 153619:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3804.565487] LNet: Removed LNI 192.168.204.154@tcp [ 3805.088265] Key type .llcrypt unregistered [ 3805.089696] Key type ._llcrypt unregistered [ 3818.735047] Key type ._llcrypt registered [ 3818.736921] Key type .llcrypt registered [ 3818.798213] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing load_modules_local [ 3819.305991] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3819.318108] alg: No test for adler32 (adler32-zlib) [ 3820.262783] Lustre: Lustre: Build Version: 2.16.61_1_gaa570bc [ 3820.381270] LNet: Added LNI 192.168.204.154@tcp [8/256/0/180] [ 3821.991245] Key type lgssc registered [ 3822.499696] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3825.932789] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3829.201951] Lustre: DEBUG MARKER: == conf-sanity test 100: check lshowmount lists MGS, MDT, OST and 0@lo ========================================================== 20:41:55 (1763084515) [ 3836.167523] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing set_hostid [ 3840.586984] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing load_modules_local [ 3844.898098] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3847.999683] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3851.028114] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3854.155460] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3857.668356] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3857.694756] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3857.703564] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3858.838080] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3858.858332] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3858.909171] Lustre: lustre-MDT0000: new disk, initializing [ 3858.953236] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3858.963965] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3860.745618] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing set_default_debug -1 all [ 3865.653992] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3865.685905] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3865.728614] Lustre: 157815:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 3865.755203] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 3865.757157] Lustre: Skipped 1 previous similar message [ 3865.798863] Lustre: lustre-MDT0001: new disk, initializing [ 3865.841306] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3865.861144] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 3865.864922] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3867.568965] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing set_default_debug -1 all [ 3871.654240] Lustre: DEBUG MARKER: oleg454-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 3873.691557] Lustre: DEBUG MARKER: oleg454-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 3877.122656] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3877.169748] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3877.312084] Lustre: lustre-OST0000: new disk, initializing [ 3877.314768] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3877.356988] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3877.502698] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 3877.507570] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 3877.561246] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3880.103368] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing set_default_debug -1 all [ 3883.881127] Lustre: DEBUG MARKER: oleg454-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3888.099403] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3888.100923] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3888.103479] LustreError: Skipped 1 previous similar message [ 3888.110388] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3888.114340] Lustre: Skipped 1 previous similar message [ 3893.221341] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3893.223491] Lustre: Skipped 1 previous similar message [ 3894.125633] LustreError: 160401:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 3894.163568] Lustre: server umount lustre-OST0000 complete [ 3896.799959] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3896.802374] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3896.806340] Lustre: Skipped 1 previous similar message [ 3896.808104] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3898.337160] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3901.635721] LustreError: 160605:0:(obd_class.h:479:obd_check_dev()) Device 22 not setup [ 3901.637982] LustreError: 160605:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 3901.669927] Lustre: server umount lustre-MDT0000 complete [ 3903.231506] LustreError: 157808:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763084590 with bad export cookie 11037468276950946935 [ 3903.235377] LustreError: 157808:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 3903.235665] LustreError: MGC192.168.204.154@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3903.319182] LustreError: 160807:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 3903.321428] LustreError: 160807:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 3903.391388] Lustre: server umount lustre-MDT0001 complete [ 3907.235472] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing unload_modules_local [ 3908.425993] Key type lgssc unregistered [ 3908.562290] LNet: 161338:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3908.564866] LNetError: 161338:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3908.573285] LNet: Removed LNI 192.168.204.154@tcp [ 3908.926192] Key type .llcrypt unregistered [ 3908.927780] Key type ._llcrypt unregistered [ 3920.232687] Key type ._llcrypt registered [ 3920.233888] Key type .llcrypt registered [ 3920.274363] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing load_modules_local [ 3920.602000] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3920.653069] alg: No test for adler32 (adler32-zlib) [ 3921.519378] Lustre: Lustre: Build Version: 2.16.61_1_gaa570bc [ 3921.612633] LNet: Added LNI 192.168.204.154@tcp [8/256/0/180] [ 3923.199156] Key type lgssc registered [ 3923.642015] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3926.675979] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3926.684051] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3927.878699] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3929.348485] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing set_default_debug -1 all [ 3932.614854] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3932.756977] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3934.141726] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing set_default_debug -1 all [ 3936.248928] Lustre: DEBUG MARKER: oleg454-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 3937.805653] Lustre: DEBUG MARKER: oleg454-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 3940.657988] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3940.819137] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3943.158196] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing set_default_debug -1 all [ 3945.345630] Lustre: DEBUG MARKER: oleg454-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3954.152851] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:3 to 0x280000401:33) [ 3958.343447] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 3958.352673] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3958.358247] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 3958.361969] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3958.366913] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 3958.732896] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3959.136348] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 3959.140178] Lustre: Skipped 1 previous similar message [ 3959.148057] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3959.153384] Lustre: Skipped 1 previous similar message [ 3959.155577] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 3959.158681] Lustre: Skipped 1 previous similar message [ 3959.160583] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3959.165318] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 3959.167627] Lustre: Skipped 1 previous similar message [ 3959.513595] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3959.875102] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3960.233203] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 3960.235978] Lustre: Skipped 2 previous similar messages [ 3960.243478] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3960.248942] Lustre: Skipped 2 previous similar messages [ 3960.250927] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 3960.254153] Lustre: Skipped 2 previous similar messages [ 3960.256287] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3960.261970] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 3960.265930] Lustre: Skipped 2 previous similar messages [ 3960.616476] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3961.004505] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3961.368031] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3961.716075] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3962.142493] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3962.503689] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 3962.507318] Lustre: Skipped 5 previous similar messages [ 3962.514164] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3962.518968] Lustre: Skipped 5 previous similar messages [ 3962.520824] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 3962.523576] Lustre: Skipped 5 previous similar messages [ 3962.526087] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3962.530810] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 3962.534191] Lustre: Skipped 5 previous similar messages [ 3962.883412] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3963.253015] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3963.620277] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3964.014653] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3964.380893] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3964.739878] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3965.092090] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3965.464985] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3965.853017] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3966.200659] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3966.536088] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 3966.540544] Lustre: Skipped 10 previous similar messages [ 3966.547982] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3966.554405] Lustre: Skipped 10 previous similar messages [ 3966.556828] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 3966.559718] Lustre: Skipped 10 previous similar messages [ 3966.561725] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3966.567056] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 3966.569947] Lustre: Skipped 10 previous similar messages [ 3966.922822] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3967.288393] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3967.636465] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3968.011251] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3968.377400] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3968.716192] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3969.098685] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3969.468025] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3969.835684] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3970.186154] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3970.519812] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3970.866564] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3971.212786] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3971.526612] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3971.844305] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3972.192057] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3972.558872] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3972.949079] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3973.323939] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3973.707335] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3974.067119] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3974.434799] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3974.810786] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 3974.813764] Lustre: Skipped 22 previous similar messages [ 3974.820713] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3974.825783] Lustre: Skipped 22 previous similar messages [ 3974.827880] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 3974.831320] Lustre: Skipped 22 previous similar messages [ 3974.833553] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3974.839086] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 3974.841561] Lustre: Skipped 22 previous similar messages [ 3975.195709] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3975.592471] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3975.981135] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3976.356524] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3976.738578] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3977.102463] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3977.470888] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3977.845527] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3978.234399] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3978.555144] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3978.896973] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3979.269713] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3979.653559] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3979.998470] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3980.353313] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3980.721400] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3981.093554] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3981.470134] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3981.842531] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3982.203965] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3982.550724] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3982.886954] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3983.237397] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3983.592424] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3983.962639] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3984.312378] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3984.646910] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3984.987928] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3985.315245] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3985.632257] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3985.970541] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3986.288029] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3986.622811] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3986.977927] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3987.334260] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3987.656884] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3987.982059] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3988.313861] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3988.672984] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3989.024890] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3989.357677] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3989.740233] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3990.116656] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3990.499470] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3990.875808] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 3990.879750] Lustre: Skipped 44 previous similar messages [ 3990.887044] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3990.890990] Lustre: Skipped 44 previous similar messages [ 3990.893427] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 3990.896578] Lustre: Skipped 44 previous similar messages [ 3990.898805] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3990.903960] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 3990.906816] Lustre: Skipped 44 previous similar messages [ 3991.273780] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3991.668082] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3992.061708] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3992.435326] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3992.795257] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3993.172223] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3993.543101] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3993.908640] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3994.289836] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3994.726632] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3995.092977] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3995.449961] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3995.804381] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3996.228727] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3996.603275] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3996.974721] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3997.356645] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3997.752833] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3998.127920] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3998.530825] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3998.935935] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3999.324567] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3999.721722] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4000.145471] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4000.548595] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4000.922522] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4001.278375] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4001.696343] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4002.122372] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4002.512928] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4002.841373] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4003.191643] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4003.533860] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4003.885351] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4004.238131] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4004.592823] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4004.951884] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4005.304266] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4005.667463] LustreError: 162729:0:(osp_precreate.c:651:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 4005.672794] LustreError: 162729:0:(osp_precreate.c:1355:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 4005.673257] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4006.045878] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4006.414817] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4006.753247] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4007.103956] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4007.453365] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4007.774698] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4008.124807] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4008.461117] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4008.795019] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4009.120065] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4009.483339] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4009.826252] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4010.169673] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4010.522790] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4010.863162] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4011.212476] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4011.558581] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4011.930571] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4012.290178] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4012.672658] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4013.038858] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4013.400096] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4013.752480] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4014.102666] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4014.464684] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4014.825503] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4015.193239] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4015.578900] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4015.955161] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4016.337194] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4016.731983] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4017.133263] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4017.502284] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4017.881470] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4018.287339] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4018.679665] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4019.064537] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4019.496621] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4019.970323] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4020.403753] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4020.848759] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4021.295873] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4021.716154] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4022.127893] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4022.524080] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4022.917835] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4022.921329] Lustre: Skipped 84 previous similar messages [ 4022.928992] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4022.935199] Lustre: Skipped 84 previous similar messages [ 4022.937509] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4022.940196] Lustre: Skipped 84 previous similar messages [ 4022.942161] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4022.947408] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4022.950391] Lustre: Skipped 84 previous similar messages [ 4023.305756] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4023.661987] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4024.046051] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4024.416461] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4024.782804] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4025.159388] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4025.558056] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4025.966310] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4026.338210] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4026.699196] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4027.074595] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4027.474666] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4027.878360] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4028.288359] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4028.697237] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4029.137887] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4029.551257] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4029.944217] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4030.338499] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4030.731360] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4031.116381] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4031.481374] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4031.850301] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4032.208881] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4032.589733] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4032.956403] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4033.343300] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4033.707429] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4034.068230] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4034.425790] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4034.782034] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4035.178639] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4035.574355] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4035.970218] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4036.378683] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4036.764681] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4037.134355] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4037.521578] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4037.897733] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4038.275038] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4038.671612] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4039.049928] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4039.434117] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4039.844247] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4040.244241] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4040.637416] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4041.043512] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4041.427506] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4041.807568] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4042.205981] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4042.591978] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4042.972757] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4043.358562] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4043.728580] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4044.094053] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4044.458416] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4044.836141] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4045.204959] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4045.597948] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4046.032331] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4046.448527] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4046.834713] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4047.218966] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4047.579067] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4047.983333] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4048.365615] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4048.734545] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4049.110141] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4049.498979] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4049.883539] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4050.264659] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4050.665978] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4051.035661] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4051.421540] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4051.804453] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4052.144481] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4052.533673] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4052.891598] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4053.242296] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4053.620586] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4054.007908] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4054.373452] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4054.735266] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4055.114618] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4055.481097] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4055.893591] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4056.282239] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4056.663610] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4057.035786] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4057.403352] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4057.766135] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4058.119503] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4058.489332] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4058.854309] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4059.221370] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4059.574095] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4059.926372] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4060.293178] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4060.662728] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4061.036343] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4061.436320] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4061.836222] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4062.194194] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4062.559391] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4062.920323] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4063.269841] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4063.641578] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4063.997336] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4064.347210] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4064.697087] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4065.079140] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4065.445162] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4065.800926] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4066.154289] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4066.558187] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4066.940196] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4067.312911] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4067.672371] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4068.064450] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4068.470226] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4068.831947] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4069.188279] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4069.556296] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4069.929022] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4070.287377] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4070.635342] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4071.010392] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4071.402655] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4071.788110] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4072.142771] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4072.483043] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4072.815488] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4073.169488] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4073.509252] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4073.861239] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4074.201650] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4074.531690] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4074.867738] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4075.202101] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4075.556057] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4075.902249] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4076.280606] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4076.662781] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4077.037982] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4077.401859] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4077.762075] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4078.135953] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4078.495162] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4078.869825] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4079.243160] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4079.605616] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4079.958513] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4080.320218] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4080.675972] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4081.069756] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4081.501246] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4081.892652] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4082.273445] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4082.654618] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4083.039692] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4083.417167] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4083.785918] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4084.141555] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4084.501252] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4084.871555] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4085.229573] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4085.594578] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4085.934135] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4086.282058] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4086.631448] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4086.991709] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4086.994786] Lustre: Skipped 170 previous similar messages [ 4087.003306] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4087.009031] Lustre: Skipped 170 previous similar messages [ 4087.011623] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4087.015558] Lustre: Skipped 170 previous similar messages [ 4087.017960] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4087.023145] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4087.026129] Lustre: Skipped 170 previous similar messages [ 4087.374396] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4087.750246] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4088.129399] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4088.545107] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4088.897988] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4089.251540] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4089.671544] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4090.029165] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4090.390968] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4090.756922] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4091.138937] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4091.519065] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4091.906557] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4092.301186] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4092.676550] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4093.041681] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4093.429384] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4093.795614] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4094.201724] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4094.656435] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4095.044808] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4095.394734] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4095.749863] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4096.120610] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4096.487656] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4096.873827] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4097.256333] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4097.666774] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4098.041260] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4098.410829] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4098.829464] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4099.201881] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4099.586715] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4099.978143] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4100.356753] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4100.741938] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4101.115714] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4101.503043] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4101.871095] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4102.241601] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4102.620607] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4103.007615] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4103.399915] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4103.815360] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4104.215747] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4104.650891] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4105.020422] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4105.404602] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4105.794435] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4106.170604] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4106.511207] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4106.869909] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4107.233533] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4107.588692] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4107.967018] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4108.339967] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4108.739214] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4109.181294] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4109.576741] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4109.953580] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4110.357050] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4110.747223] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4111.147672] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4111.543599] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4111.884460] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4112.221447] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4112.563352] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4112.933971] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4113.275197] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4113.620936] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4113.988638] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4114.374508] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4114.714089] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4115.056827] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4115.393597] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4115.745911] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4116.089221] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4116.443738] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4116.798862] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4117.169236] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4117.537179] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4117.897693] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4118.274557] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4118.662241] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4119.053798] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4119.435541] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4119.823550] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4120.185319] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4120.568424] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4120.946521] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4121.318089] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4121.680700] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4122.039531] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4122.448398] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4122.849227] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4123.244860] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4123.609663] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4123.968123] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4124.320730] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4124.699387] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4125.051687] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4125.426716] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4125.774033] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4126.153616] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4126.539114] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4126.894991] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4127.249267] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4127.643165] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4128.018882] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4128.384361] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4128.746050] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4129.130286] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4129.502758] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4129.898171] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4130.287865] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4130.671226] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4131.034537] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4131.427792] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4131.797344] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4132.177564] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4132.584732] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4132.979681] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4133.390525] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4133.769264] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4134.141376] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4134.504171] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4134.880280] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4135.259819] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4135.618588] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4135.991319] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4136.384131] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4136.763921] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4137.136654] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4137.522686] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4137.900046] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4138.270446] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4138.651951] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4139.039968] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4139.399821] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4139.768987] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4140.175721] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4140.552453] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4140.922251] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4141.322295] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4141.701069] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4142.079977] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4142.460639] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4142.850294] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4143.209858] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4143.586602] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4143.956972] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4144.327482] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4144.694790] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4145.076435] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4145.471564] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4145.838704] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4146.197445] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4146.569470] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4146.954655] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4147.314819] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4147.673269] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4148.031847] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4148.399217] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4148.776088] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4149.168902] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4149.540689] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4149.924805] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4150.295376] LustreError: 162729:0:(osp_precreate.c:651:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 4150.299908] LustreError: 162729:0:(osp_precreate.c:1355:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 4150.300829] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4150.647677] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4151.013269] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4151.374939] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4151.754986] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4152.126501] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4152.509877] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4152.859245] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4153.224978] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4153.598077] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4153.958086] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4154.301285] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4154.655109] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4155.012647] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4155.361316] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4155.695156] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4156.057282] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4156.403948] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4156.743288] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4157.099884] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4157.460483] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4157.807143] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4158.151203] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4158.519113] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4158.871207] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4159.255113] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4159.636436] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4160.004925] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4160.367207] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4160.718301] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4161.090744] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4161.443257] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4161.802299] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4162.163954] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4162.522033] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4162.885311] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4163.268520] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4163.621584] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4164.012165] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4164.370839] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4164.742593] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4165.087183] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4165.442507] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4165.786297] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4166.150180] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4166.509849] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4166.875416] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4167.253954] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4167.637653] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4168.003551] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4168.364409] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4168.776678] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4169.158470] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4169.543855] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4169.897883] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4170.235565] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4170.615596] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4170.948201] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4171.266031] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4171.595402] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4171.942692] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4172.277940] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4172.606665] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4172.945667] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4173.299518] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4173.617946] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4173.982434] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4174.331346] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4174.684434] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4175.022726] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4175.376586] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4175.688849] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4176.037945] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4176.353689] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4176.686528] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4177.046149] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4177.405179] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4177.761972] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4178.119401] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4178.475370] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4178.810887] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4179.145617] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4179.488833] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4179.838199] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4180.197883] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4180.542450] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4180.880619] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4181.208729] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4181.556259] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4181.899408] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4182.265251] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4182.628034] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4182.981618] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4183.321602] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4183.677543] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4184.028489] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4184.402346] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4184.735397] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4185.088429] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4185.427907] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4185.764570] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4186.106989] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4186.460905] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4186.803218] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4187.144474] LustreError: 162729:0:(osp_precreate.c:651:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 4187.149060] LustreError: 162729:0:(osp_precreate.c:1355:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 4187.150109] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4187.519477] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4187.883447] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4188.244811] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4188.583397] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4188.922209] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4189.283760] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4189.639140] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4190.010070] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4190.385386] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4190.742088] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4191.076483] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4191.406456] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4191.768561] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4192.138415] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4192.495589] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4192.853405] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4193.219481] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4193.583589] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4193.952028] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4194.325862] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4194.690073] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4195.054978] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4195.410269] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4195.774387] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4196.157070] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4196.528706] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4196.899234] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4197.290820] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4197.671632] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4198.049023] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4198.431330] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4198.836566] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4199.211214] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4199.592802] LustreError: 162729:0:(osp_precreate.c:651:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 4199.598274] LustreError: 162729:0:(osp_precreate.c:1355:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 4199.598796] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4199.950039] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4200.321452] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4200.730628] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4201.121892] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4201.539381] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4201.989567] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4202.413474] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4202.833842] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4203.221844] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4203.592545] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4203.957685] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4204.299940] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4204.655636] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4205.001644] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4205.354491] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4205.704649] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4206.048620] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4206.392540] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4206.740163] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4207.084078] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4207.432624] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4207.784600] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4208.160705] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4208.511901] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4208.879635] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4209.240502] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4209.599259] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4209.992965] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4210.346792] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4210.708242] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4211.055509] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4211.416240] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4211.773666] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4212.179404] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4212.533778] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4212.907989] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4213.300635] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4213.680871] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4214.039299] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4214.414600] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4214.790424] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4215.172427] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4215.174601] Lustre: Skipped 347 previous similar messages [ 4215.180801] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4215.185666] Lustre: Skipped 347 previous similar messages [ 4215.187329] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4215.190466] Lustre: Skipped 347 previous similar messages [ 4215.192439] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4215.197040] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4215.199371] Lustre: Skipped 347 previous similar messages [ 4215.541214] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4215.881995] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4216.247853] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4216.591141] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4216.917532] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4217.255923] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4217.579098] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4217.917934] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4218.261464] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4218.592826] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4218.924038] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4219.290386] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4219.647962] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4220.064896] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4220.436085] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4220.834241] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4221.198411] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4221.602779] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4221.991212] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4222.381757] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4222.770551] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4223.139774] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4223.511444] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4223.911861] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4224.280976] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4224.672850] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4225.080158] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4225.493222] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4225.909367] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4226.314257] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4226.689210] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4227.053623] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4227.426783] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4227.831090] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4228.216449] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4228.623899] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4228.984500] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4229.330495] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4229.682494] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4230.033155] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4230.389249] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4230.735037] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4231.094056] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4231.495727] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4231.884341] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4232.292159] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4232.687827] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4233.103873] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4233.496390] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4233.855212] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4234.239586] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4234.605281] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4234.961308] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4235.359686] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4235.727294] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4236.106492] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4236.453553] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4236.802531] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4237.128611] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4237.477436] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4237.838170] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4238.212855] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4238.600667] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4238.957879] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4239.344607] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4239.727704] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4240.105468] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4240.475240] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4240.836319] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4241.196670] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4241.577591] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4241.937347] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4242.314000] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4242.702758] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4243.100404] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4243.509610] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4243.867926] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4244.217408] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4244.563756] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4244.905081] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4245.270414] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4245.652312] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4246.010189] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4246.370651] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4246.720680] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4247.090659] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4247.442083] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4247.795950] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4248.115659] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4248.452328] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4248.772704] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4249.111729] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4249.440300] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4249.786682] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4250.118320] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4250.469881] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4250.808884] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4251.138807] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4251.470737] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4251.810582] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4252.156241] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4252.502095] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4252.853460] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4253.193918] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4253.537663] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4253.893366] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4254.223041] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4254.551727] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4254.902350] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4255.250979] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4255.612677] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4255.970542] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4256.334510] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4256.697079] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4257.085078] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4257.466794] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4257.800413] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4258.146802] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4258.470735] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4258.821889] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4259.144240] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4259.467540] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4259.789400] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4260.147857] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4260.493274] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4260.843472] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4261.167285] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4261.519813] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4261.827180] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4262.150478] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4262.488760] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4262.822847] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4263.150020] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4263.475600] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4263.812304] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4264.177310] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4264.518198] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4264.857964] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4265.183067] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4265.511585] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4265.831071] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4266.153653] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4266.471882] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4266.786555] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4267.116281] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4267.440084] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4267.759258] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4268.098594] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4268.431402] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4268.770064] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4269.101170] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4269.432762] LustreError: 162729:0:(osp_precreate.c:651:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 4269.436732] LustreError: 162729:0:(osp_precreate.c:1355:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 4269.437060] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4269.782205] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4270.111966] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4270.439376] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4270.789152] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4271.114957] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4271.454880] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4271.779450] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4272.102489] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4272.417065] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4272.734815] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4273.052333] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4273.390626] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4273.699776] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4274.037020] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4274.360196] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4274.703079] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4275.043740] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4275.388197] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4275.750179] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4276.121107] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4276.472706] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4276.848317] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4277.209188] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4277.576259] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4277.954408] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4278.326477] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4278.669454] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4279.027745] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4279.384356] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4279.735239] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4280.076377] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4280.415042] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4280.759664] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4281.146885] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4281.526271] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4281.898170] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4282.262787] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4282.596595] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4282.963774] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4283.323137] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4283.680133] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4284.002844] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4284.327348] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4284.655310] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4284.962450] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4285.291681] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4285.643688] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4285.994027] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4286.338591] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4286.681016] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4287.032948] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4287.401174] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4287.754263] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4288.102986] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4288.460790] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4288.790223] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4289.120287] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4289.451204] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4289.809828] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4290.138158] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4290.477303] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4290.830216] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4291.164755] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4291.501947] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4291.831285] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4292.183510] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4292.520196] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4292.845815] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4293.176884] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4293.500270] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4293.810818] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4294.129070] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4294.484610] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4294.863638] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4295.198898] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4295.562854] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4295.910938] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4296.255238] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4296.596650] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4296.952333] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4297.312732] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4297.645904] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4298.003144] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4298.346805] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4298.678483] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4299.003837] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4299.324879] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4299.654517] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4299.980741] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4300.305614] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4300.618518] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4300.947954] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4301.277772] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4301.613421] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4301.969057] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4302.311654] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4302.717939] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4303.082397] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4303.450759] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4303.839453] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4304.206310] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4304.557697] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4304.937224] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4305.309738] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4305.654482] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4306.018083] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4306.402798] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4306.757049] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4307.135986] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4307.505889] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4307.877242] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4308.217676] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4308.582912] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4308.935146] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4309.300519] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4309.646871] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4310.009922] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4310.374928] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4310.736330] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4311.089175] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4311.447990] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4311.801164] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4312.159717] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4312.543854] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4312.953059] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4313.327749] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4313.702510] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4314.094164] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4314.473533] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4314.814487] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4315.169711] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4315.534448] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4315.934491] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4316.332882] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4316.707208] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4317.057747] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4317.397889] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4317.754398] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4318.111410] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4318.454098] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4318.794712] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4319.153795] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4319.521260] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4319.893844] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4320.229222] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4320.569122] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4320.909841] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4321.235378] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4321.582279] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4321.926332] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4322.310543] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4322.668386] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4323.022749] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4484.525353] LustreError: 164306:0:(ofd_dev.c:1780:ofd_create_hdl()) lustre-OST0000: unable to precreate: rc = -28 [ 4484.529154] LustreError: 162729:0:(osp_precreate.c:651:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -28 [ 4613.088112] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4613.092620] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4613.098201] Lustre: Skipped 305 previous similar messages [ 4613.101046] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4613.603182] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4613.609840] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4614.624374] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4614.627109] Lustre: Skipped 1 previous similar message [ 4619.744338] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4619.746942] Lustre: Skipped 1 previous similar message [ 4624.867498] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4624.870064] Lustre: Skipped 1 previous similar message [ 4625.375864] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 4625.411734] LustreError: 216491:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 4625.505070] Lustre: server umount lustre-OST0000 complete [ 4628.962200] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4635.104270] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4635.106990] Lustre: Skipped 4 previous similar messages [ 4641.247172] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 4641.293711] LustreError: 216695:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup [ 4641.296222] LustreError: 216695:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 4641.378933] Lustre: server umount lustre-MDT0000 complete [ 4642.860890] LustreError: 162689:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763085330 with bad export cookie 5061896695502069624 [ 4642.863650] LustreError: MGC192.168.204.154@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4642.866216] LustreError: 162689:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4642.939151] LustreError: 216897:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [ 4642.941429] LustreError: 216897:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 4643.002704] Lustre: server umount lustre-MDT0001 complete [ 4647.160093] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing unload_modules_local [ 4648.438773] Key type lgssc unregistered [ 4648.594441] LNet: 217428:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4648.598469] LNetError: 217428:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4648.606524] LNet: Removed LNI 192.168.204.154@tcp [ 4649.009492] Key type .llcrypt unregistered [ 4649.011553] Key type ._llcrypt unregistered [ 4660.191205] Key type ._llcrypt registered [ 4660.192894] Key type .llcrypt registered [ 4660.237687] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing load_modules_local [ 4660.661842] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4660.707994] alg: No test for adler32 (adler32-zlib) [ 4661.594725] Lustre: Lustre: Build Version: 2.16.61_1_gaa570bc [ 4661.711347] LNet: Added LNI 192.168.204.154@tcp [8/256/0/180] [ 4663.311773] Key type lgssc registered [ 4663.816181] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4667.214042] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4667.223654] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4668.444840] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4670.072914] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing set_default_debug -1 all [ 4673.715053] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4673.896650] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4675.528501] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing set_default_debug -1 all [ 4677.832454] Lustre: DEBUG MARKER: oleg454-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 4679.377729] Lustre: DEBUG MARKER: oleg454-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 4682.230480] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4682.377884] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4684.843929] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing set_default_debug -1 all [ 4687.187796] Lustre: DEBUG MARKER: oleg454-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4692.654365] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49666 to 0x280000401:49697) [ 4695.438539] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4696.157427] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4696.163907] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4696.167150] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4696.171454] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4696.171940] LustreError: 218819:0:(osp_precreate.c:1263:osp_precreate_thread()) cfs_fail_timeout id 2107 sleeping for 20000ms [ 4698.084559] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4698.085459] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4698.092775] Lustre: Skipped 1 previous similar message [ 4698.098184] Lustre: Skipped 1 previous similar message [ 4698.697876] Lustre: lustre-OST0000: Not available for connect from 192.168.204.54@tcp (stopping) [ 4703.030093] LustreError: 221197:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 4703.087360] Lustre: server umount lustre-OST0000 complete [ 4716.247219] LustreError: 218819:0:(osp_precreate.c:1263:osp_precreate_thread()) cfs_fail_timeout id 2107 awake [ 4732.294850] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4732.452334] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4734.973292] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing set_default_debug -1 all [ 4737.128129] Lustre: DEBUG MARKER: oleg454-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4738.597724] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4738.602899] LustreError: lustre-OST0000-osc-MDT0001: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4738.608474] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4739.780300] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid in FULL state after 2 sec [ 4741.170646] Lustre: DEBUG MARKER: oleg454-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff9587cd13f800.ost_server_uuid 50 [ 4741.774541] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff9587cd13f800.ost_server_uuid in FULL state after 0 sec [ 4743.649386] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4743.649735] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4743.652197] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4743.655446] Lustre: Skipped 1 previous similar message [ 4748.769654] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4748.772577] Lustre: Skipped 1 previous similar message [ 4748.972174] LustreError: 222515:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 4748.974692] LustreError: 222515:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 4749.012705] Lustre: server umount lustre-OST0000 complete [ 4750.816171] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4750.820145] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4753.889828] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4753.894183] Lustre: Skipped 3 previous similar messages [ 4756.544451] LustreError: 222716:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup [ 4756.547229] LustreError: 222716:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 4756.598690] Lustre: server umount lustre-MDT0000 complete [ 4758.157463] LustreError: 218782:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763085445 with bad export cookie 15109995381171759857 [ 4758.160579] LustreError: MGC192.168.204.154@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4758.162379] LustreError: 218782:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4758.316175] Lustre: server umount lustre-MDT0001 complete [ 4762.536074] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing unload_modules_local [ 4763.781397] Key type lgssc unregistered [ 4763.935392] LNet: 223448:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4763.938107] LNetError: 223448:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4763.945491] LNet: Removed LNI 192.168.204.154@tcp [ 4764.299716] Key type .llcrypt unregistered [ 4764.301334] Key type ._llcrypt unregistered [ 4777.043951] Key type ._llcrypt registered [ 4777.045624] Key type .llcrypt registered [ 4777.086044] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing load_modules_local [ 4777.536224] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4777.550483] alg: No test for adler32 (adler32-zlib) [ 4778.418519] Lustre: Lustre: Build Version: 2.16.61_1_gaa570bc [ 4778.514763] LNet: Added LNI 192.168.204.154@tcp [8/256/0/180] [ 4780.103377] Key type lgssc registered [ 4780.615217] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4784.612128] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4784.621148] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4785.863754] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4787.443773] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing set_default_debug -1 all [ 4791.007690] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4791.189447] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4792.834432] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing set_default_debug -1 all [ 4794.034917] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 4796.705109] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4796.836915] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4799.139248] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing set_default_debug -1 all [ 4802.626045] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4802.651807] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4802.722462] Lustre: lustre-OST0001: new disk, initializing [ 4802.725507] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 4802.774912] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 4804.623762] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 4804.628687] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 4804.673356] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 4805.151260] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing set_default_debug -1 all [ 4810.803778] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49699 to 0x280000401:49729) [ 4811.343871] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 4814.477312] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 4831.200883] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4831.201982] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4831.204967] Lustre: Skipped 3 previous similar messages [ 4831.206932] Lustre: Skipped 2 previous similar messages [ 4833.550429] LustreError: 228496:0:(obd_class.h:479:obd_check_dev()) Device 28 not setup [ 4833.588631] Lustre: server umount lustre-MDT0000 complete [ 4834.984613] LustreError: 225115:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763085522 with bad export cookie 17346084230846029015 [ 4834.987660] LustreError: MGC192.168.204.154@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4834.987932] LustreError: 225115:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4835.058552] LustreError: 228698:0:(obd_class.h:479:obd_check_dev()) Device 27 not setup [ 4835.061135] LustreError: 228698:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 4835.134540] Lustre: server umount lustre-MDT0001 complete [ 4845.686276] LustreError: 228899:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 4845.688318] LustreError: 228899:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 4845.708941] Lustre: server umount lustre-OST0000 complete [ 4857.329665] LustreError: 229101:0:(obd_class.h:479:obd_check_dev()) Device 2 not setup [ 4857.332499] LustreError: 229101:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 4857.381088] Lustre: server umount lustre-OST0001 complete [ 4858.555252] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro [ 4858.587655] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4859.038968] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro [ 4859.076397] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4859.470236] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: ,errors=remount-ro [ 4859.503406] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4859.896196] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: ,errors=remount-ro [ 4859.926698] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4864.914611] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing load_modules_local [ 4868.434749] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4868.597665] LustreError: 230509:0:(ldlm_lib.c:1179:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4868.605031] LustreError: 230509:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 4868.623764] Lustre: mylustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4870.060362] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing set_default_debug -1 all [ 4873.027714] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4874.451591] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing set_default_debug -1 all [ 4875.444780] Lustre: 231618:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 4877.755186] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4877.875140] Lustre: mylustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4877.877875] Lustre: Skipped 1 previous similar message [ 4878.883480] LustreError: 231974:0:(ldlm_lib.c:1179:target_handle_connect()) mylustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4878.887073] Lustre: mylustre-OST0000: new connection from mylustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49731 to 0x280000401:49761) [ 4879.894973] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing set_default_debug -1 all [ 4882.976516] LustreError: 232159:0:(ldlm_lib.c:1179:target_handle_connect()) mylustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4882.982693] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4882.984069] LustreError: 232159:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 4885.046039] Lustre: mylustre-OST0001: new connection from mylustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:33) [ 4885.262835] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing set_default_debug -1 all [ 4888.507915] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 4889.751114] Lustre: 233463:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 4903.906948] Lustre: mylustre-MDT0000-lwp-OST0001: Connection to mylustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4903.908159] LustreError: mylustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4903.909404] Lustre: mylustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4903.912020] Lustre: Skipped 2 previous similar messages [ 4905.279665] LustreError: 233769:0:(obd_class.h:479:obd_check_dev()) Device 10 not setup [ 4905.281774] LustreError: 233769:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 4905.311148] Lustre: server umount mylustre-MDT0000 complete [ 4906.618119] LustreError: 230484:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763085594 with bad export cookie 17346084230846031780 [ 4906.621358] LustreError: MGC192.168.204.154@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4906.622858] LustreError: 230484:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4906.739059] Lustre: server umount mylustre-MDT0001 complete [ 4918.196116] LustreError: 234170:0:(obd_class.h:479:obd_check_dev()) Device 23 not setup [ 4918.198193] LustreError: 234170:0:(obd_class.h:479:obd_check_dev()) Skipped 15 previous similar messages [ 4918.216564] Lustre: server umount mylustre-OST0000 complete [ 4928.728457] Lustre: server umount mylustre-OST0001 complete [ 4929.779893] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro [ 4929.811535] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4930.171217] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro [ 4930.207633] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4930.581794] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: ,errors=remount-ro [ 4930.608266] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4930.969171] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: ,errors=remount-ro [ 4930.999907] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4935.757574] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing load_modules_local [ 4939.289502] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4939.450051] LustreError: 235782:0:(ldlm_lib.c:1179:target_handle_connect()) tfs-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4939.474713] Lustre: tfs-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4939.476946] Lustre: Skipped 1 previous similar message [ 4940.794473] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing set_default_debug -1 all [ 4943.640645] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4945.057626] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing set_default_debug -1 all [ 4946.038861] Lustre: 236891:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 4948.531436] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4949.729745] LustreError: 237408:0:(ldlm_lib.c:1179:target_handle_connect()) tfs-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4949.735751] Lustre: tfs-OST0000: new connection from tfs-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49731 to 0x280000401:49793) [ 4950.871586] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing set_default_debug -1 all [ 4953.845898] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4955.323584] Lustre: tfs-OST0001: new connection from tfs-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:35 to 0x2c0000401:65) [ 4955.980402] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing set_default_debug -1 all [ 4959.137880] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 4960.273598] Lustre: 238737:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 4969.442198] Lustre: tfs-MDT0000-lwp-MDT0001: Connection to tfs-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4969.445957] Lustre: Skipped 1 previous similar message [ 4969.447427] LustreError: tfs-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4969.449050] Lustre: tfs-MDT0000: Not available for connect from 0@lo (stopping) [ 4969.450895] Lustre: Skipped 6 previous similar messages [ 4973.310490] LustreError: 238984:0:(obd_class.h:479:obd_check_dev()) Device 10 not setup [ 4973.313075] LustreError: 238984:0:(obd_class.h:479:obd_check_dev()) Skipped 9 previous similar messages [ 4973.345812] Lustre: server umount tfs-MDT0000 complete [ 4974.561320] LustreError: 236513:0:(ldlm_lib.c:1179:target_handle_connect()) tfs-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4974.565616] LustreError: 236513:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 5 previous similar messages [ 4974.645344] LustreError: 235758:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763085662 with bad export cookie 17346084230846033670 [ 4974.648720] LustreError: MGC192.168.204.154@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4974.650021] LustreError: 235758:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4974.762568] Lustre: server umount tfs-MDT0001 complete [ 4985.858620] Lustre: server umount tfs-OST0000 complete [ 4997.344621] Lustre: server umount tfs-OST0001 complete [ 4998.452593] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro [ 4998.489288] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4998.841266] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro [ 4998.869412] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4999.234180] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: ,errors=remount-ro [ 4999.251933] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4999.611330] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: ,errors=remount-ro [ 4999.642564] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5004.488513] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing load_modules_local [ 5008.136219] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5008.329495] LustreError: 241007:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5008.359272] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5008.361186] Lustre: Skipped 3 previous similar messages [ 5009.760804] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing set_default_debug -1 all [ 5012.748766] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5014.229231] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing set_default_debug -1 all [ 5015.193343] Lustre: 242116:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 5017.391677] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5018.532937] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49731 to 0x280000401:49825) [ 5019.420857] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing set_default_debug -1 all [ 5022.310622] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5023.402270] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:35 to 0x2c0000401:97) [ 5024.262424] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing set_default_debug -1 all [ 5027.303764] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 5028.453492] Lustre: 243964:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 5035.588608] Lustre: DEBUG MARKER: == conf-sanity test 104a: Make sure user defined options are reflected in mount ========================================================== 21:02:02 (1763085722) [ 5048.800030] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 5048.802230] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5048.806315] Lustre: Skipped 1 previous similar message [ 5048.808629] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5053.501465] LustreError: 244331:0:(obd_class.h:479:obd_check_dev()) Device 10 not setup [ 5053.503510] LustreError: 244331:0:(obd_class.h:479:obd_check_dev()) Skipped 25 previous similar messages [ 5053.533682] Lustre: server umount lustre-MDT0000 complete [ 5054.435038] LustreError: 241007:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5054.440645] LustreError: 241007:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 4 previous similar messages [ 5054.773159] LustreError: 240982:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763085742 with bad export cookie 17346084230846035336 [ 5054.777346] LustreError: MGC192.168.204.154@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5054.777680] LustreError: 240982:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 5059.554533] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5059.555345] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 5059.557671] Lustre: Skipped 4 previous similar messages [ 5059.560257] Lustre: Skipped 4 previous similar messages [ 5069.279077] Lustre: lustre-MDT0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 5069.362688] Lustre: server umount lustre-MDT0001 complete [ 5085.151159] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 5085.193881] Lustre: server umount lustre-OST0000 complete [ 5092.559285] Lustre: server umount lustre-OST0001 complete [ 5094.548644] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing set_hostid [ 5097.612711] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing load_modules_local [ 5100.971363] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5103.005744] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5104.937742] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5106.926047] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5109.114829] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5109.135823] LDISKFS-fs (dm-0): Mount option "noacl" will be removed by 3.5 [ 5109.135823] Contact linux-ldiskfs@vger.kernel.org if you think we should keep it. [ 5109.135823] [ 5109.142406] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: acl,user_xattr,errors=remount-ro,noacl,no_mbcache,nodelalloc [ 5109.233769] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 5109.245045] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 5109.279934] Lustre: lustre-MDT0000: new disk, initializing [ 5109.306262] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5109.308660] Lustre: Skipped 3 previous similar messages [ 5109.315130] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 5110.623814] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing set_default_debug -1 all [ 5114.763500] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5114.786559] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5114.816251] Lustre: 247116:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 5114.835319] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 5114.836963] Lustre: Skipped 1 previous similar message [ 5114.869434] Lustre: lustre-MDT0001: new disk, initializing [ 5114.910562] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 5114.913728] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 5116.116751] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing set_default_debug -1 all [ 5120.300202] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5120.324067] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5120.434027] Lustre: lustre-OST0000: new disk, initializing [ 5120.436862] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 5122.038668] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 5122.041554] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 5122.079565] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 5122.441971] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing set_default_debug -1 all [ 5126.600203] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5126.626181] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5126.679299] Lustre: lustre-OST0001: new disk, initializing [ 5126.681091] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 5128.638230] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing set_default_debug -1 all [ 5128.655945] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 5128.659950] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 5128.699666] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 5132.993215] Lustre: DEBUG MARKER: == conf-sanity test 104b: Mount uses last flock argument ========================================================== 21:03:39 (1763085819) [ 5136.722085] Lustre: DEBUG MARKER: == conf-sanity test 105: check file creation for ro and rw bind mnt pt ========================================================== 21:03:43 (1763085823) [ 5149.159196] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5149.160685] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5149.164439] Lustre: Skipped 1 previous similar message [ 5149.169481] Lustre: Skipped 4 previous similar messages [ 5154.291630] LustreError: 250536:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 5154.293936] LustreError: 250536:0:(obd_class.h:479:obd_check_dev()) Skipped 25 previous similar messages [ 5154.322787] Lustre: server umount lustre-OST0000 complete [ 5159.396380] LustreError: 248657:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5159.402413] LustreError: 248657:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 4 previous similar messages [ 5162.151649] Lustre: server umount lustre-MDT0000 complete [ 5163.519424] LustreError: 247909:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763085850 with bad export cookie 17346084230846036890 [ 5163.523455] LustreError: MGC192.168.204.154@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5163.524255] LustreError: 247909:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 5168.544472] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 5168.546571] Lustre: Skipped 6 previous similar messages [ 5169.727366] Lustre: server umount lustre-MDT0001 complete [ 5173.805179] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing unload_modules_local [ 5183.653540] Lustre: server umount lustre-OST0001 complete [ 5186.020898] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing set_hostid [ 5192.731291] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing load_modules_local [ 5196.720480] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5199.354299] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5201.918299] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5204.549536] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5207.154281] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5207.178946] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5207.284399] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 5207.298541] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 5207.340304] Lustre: lustre-MDT0000: new disk, initializing [ 5207.371451] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 5208.920692] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing set_default_debug -1 all [ 5213.592539] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5213.620339] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5213.660249] Lustre: 253992:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 5213.773095] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 5215.151517] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing set_default_debug -1 all [ 5218.319869] Lustre: DEBUG MARKER: oleg454-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 5219.664319] Lustre: DEBUG MARKER: oleg454-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 5222.010567] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5222.039232] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5224.062740] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 5224.065478] Lustre: Skipped 1 previous similar message [ 5224.068040] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 5224.098465] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 5224.209211] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing set_default_debug -1 all [ 5227.314068] Lustre: DEBUG MARKER: oleg454-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 5234.657714] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 5234.658449] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5234.664833] Lustre: Skipped 4 previous similar messages [ 5234.666916] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5234.668798] Lustre: Skipped 1 previous similar message [ 5235.918730] Lustre: server umount lustre-OST0000 complete [ 5239.264042] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 5239.266913] LustreError: Skipped 1 previous similar message [ 5243.286226] Lustre: server umount lustre-MDT0000 complete [ 5244.586452] LustreError: 254773:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763085932 with bad export cookie 17346084230846038353 [ 5244.588758] LustreError: MGC192.168.204.154@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5244.591424] LustreError: 254773:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 5244.726806] Lustre: server umount lustre-MDT0001 complete [ 5248.216195] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing unload_modules_local [ 5249.194682] Key type lgssc unregistered [ 5249.322408] LNet: 257371:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5249.325398] LNetError: 257371:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5249.331328] LNet: Removed LNI 192.168.204.154@tcp [ 5249.647124] Key type .llcrypt unregistered [ 5249.648303] Key type ._llcrypt unregistered [ 5259.651542] Key type ._llcrypt registered [ 5259.652448] Key type .llcrypt registered [ 5259.694585] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing load_modules_local [ 5260.023066] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5260.096796] alg: No test for adler32 (adler32-zlib) [ 5260.956475] Lustre: Lustre: Build Version: 2.16.61_1_gaa570bc [ 5261.052947] LNet: Added LNI 192.168.204.154@tcp [8/256/0/180] [ 5262.639178] Key type lgssc registered [ 5263.051480] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5265.912425] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5265.918944] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5267.108508] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5268.623593] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing set_default_debug -1 all [ 5271.605145] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5271.735759] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5273.099302] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing set_default_debug -1 all [ 5275.057224] Lustre: DEBUG MARKER: oleg454-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 5276.312645] Lustre: DEBUG MARKER: oleg454-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 5278.678738] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5278.807600] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5280.735820] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing set_default_debug -1 all [ 5282.610767] Lustre: DEBUG MARKER: oleg454-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 5282.956645] Lustre: Setting parameter lustre-OST0000.ost.unknown_param=50 in log lustre-OST0000 [ 5287.398169] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5287.398262] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 5287.404170] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5289.707633] LustreError: 261081:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 5289.746603] Lustre: server umount lustre-OST0000 complete [ 5292.513542] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 5292.514122] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5292.522622] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5292.524700] Lustre: Skipped 1 previous similar message [ 5297.277864] LustreError: 261290:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup [ 5297.280008] LustreError: 261290:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 5297.306979] Lustre: server umount lustre-MDT0000 complete [ 5297.632432] LustreError: 258791:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5297.638554] LustreError: 258791:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 5298.637280] LustreError: 258772:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763085986 with bad export cookie 12176986217304694824 [ 5298.640809] LustreError: MGC192.168.204.154@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5298.642783] LustreError: 258772:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 5298.716173] LustreError: 261491:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [ 5298.718126] LustreError: 261491:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 5298.775083] Lustre: server umount lustre-MDT0001 complete [ 5302.636615] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing unload_modules_local [ 5303.729482] Key type lgssc unregistered [ 5303.871432] LNet: 262021:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5303.874568] LNetError: 262021:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5303.884435] LNet: Removed LNI 192.168.204.154@tcp [ 5304.184524] Key type .llcrypt unregistered [ 5304.186047] Key type ._llcrypt unregistered [ 5311.758325] Key type ._llcrypt registered [ 5311.759759] Key type .llcrypt registered [ 5311.807564] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing load_modules_local [ 5312.179118] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5312.186924] alg: No test for adler32 (adler32-zlib) [ 5313.052427] Lustre: Lustre: Build Version: 2.16.61_1_gaa570bc [ 5313.144867] LNet: Added LNI 192.168.204.154@tcp [8/256/0/180] [ 5314.727181] Key type lgssc registered [ 5315.148222] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5318.650482] Lustre: lustre-OST0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5318.657855] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5334.111398] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 5339.231275] LustreError: 263172:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.204.154@tcp: failed processing log, type 4: rc = -110 [ 5365.855209] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5367.939226] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing set_default_debug -1 all [ 5369.933415] Lustre: DEBUG MARKER: oleg454-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 5370.687907] LustreError: 263835:0:(obd_class.h:479:obd_check_dev()) Device 1 not setup [ 5370.727239] Lustre: server umount lustre-OST0000 complete [ 5376.430913] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing unload_modules_local [ 5377.607497] Key type lgssc unregistered [ 5377.762426] LNet: 264671:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5377.765126] LNetError: 264671:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5377.774343] LNet: Removed LNI 192.168.204.154@tcp [ 5378.089720] Key type .llcrypt unregistered [ 5378.091355] Key type ._llcrypt unregistered [ 5393.154040] Key type ._llcrypt registered [ 5393.155506] Key type .llcrypt registered [ 5393.198393] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing load_modules_local [ 5393.666230] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5393.676907] alg: No test for adler32 (adler32-zlib) [ 5394.541890] Lustre: Lustre: Build Version: 2.16.61_1_gaa570bc [ 5394.644536] LNet: Added LNI 192.168.204.154@tcp [8/256/0/180] [ 5396.231190] Key type lgssc registered [ 5396.609115] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5401.294595] loop0: detected capacity change from 0 to 204800000 [ 5401.356410] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 5401.360495] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 5401.495565] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 5401.517366] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5401.892551] loop0: detected capacity change from 0 to 204800000 [ 5401.958971] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 5401.963482] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 5402.105345] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 5402.124593] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5402.507268] loop0: detected capacity change from 0 to 204800000 [ 5402.573460] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 5402.578437] blk_update_request: operation not supported error, dev loop0, sector 8224 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 5402.623687] blk_update_request: operation not supported error, dev loop0, sector 58264 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 5402.671321] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5403.047660] loop0: detected capacity change from 0 to 204800000 [ 5403.111215] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 5403.196264] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5403.542654] loop0: detected capacity change from 0 to 204800000 [ 5403.556130] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 5405.053673] loop0: detected capacity change from 0 to 204800000 [ 5405.064647] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 5406.431029] loop0: detected capacity change from 0 to 204800000 [ 5406.443593] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 5407.816110] loop0: detected capacity change from 0 to 204800000 [ 5407.825855] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 5409.205568] loop0: detected capacity change from 0 to 204800000 [ 5409.246508] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5409.282515] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5409.289517] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5409.310441] Lustre: 268049:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 5409.314825] Lustre: 268049:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x10000-MDT0000' with [0x200000005:0x1:0x0]: rc = 0 [ 5409.318148] Lustre: 268049:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000003:0xc:0x0]: rc = 0 [ 5409.321564] Lustre: 268049:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x2010000-MDT0000' with [0x200000005:0x3:0x0]: rc = 0 [ 5409.325434] Lustre: 268049:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x1010000-MDT0000' with [0x200000005:0x2:0x0]: rc = 0 [ 5409.330772] Lustre: 268049:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000003:0xb:0x0]: rc = 0 [ 5409.334084] Lustre: 268049:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000003:0xa:0x0]: rc = 0 [ 5409.339637] Lustre: 268049:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index 'nodemap' with [0x200000003:0x3:0x0]: rc = 0 [ 5409.345075] Lustre: 268049:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x20000' with [0x200000006:0x20000:0x0]: rc = 0 [ 5409.348745] Lustre: 268049:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x1020000' with [0x200000006:0x1020000:0x0]: rc = 0 [ 5409.352101] Lustre: 268049:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x2020000' with [0x200000006:0x2020000:0x0]: rc = 0 [ 5409.356526] Lustre: 268049:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000006:0x2010000:0x0]: rc = 0 [ 5409.360604] Lustre: 268049:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000006:0x1010000:0x0]: rc = 0 [ 5409.364473] Lustre: 268049:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000006:0x10000:0x0]: rc = 0 [ 5411.918261] LustreError: 268315:0:(obd_class.h:479:obd_check_dev()) Device 2 not setup [ 5411.966788] Lustre: server umount MGS complete [ 5412.270843] loop0: detected capacity change from 0 to 204800000 [ 5412.316068] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5412.500538] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5412.516509] Lustre: lustre-MDT0000: Aborting client recovery [ 5412.831965] loop1: detected capacity change from 0 to 204800000 [ 5412.888712] LDISKFS-fs (loop1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5412.918275] LDISKFS-fs (loop1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5412.937433] Lustre: 268484:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 5412.940773] Lustre: 268484:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0001: restore index '0x10000-MDT0001' with [0x200000005:0x1:0x0]: rc = 0 [ 5412.945187] Lustre: 268484:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0001: restore index '0x1010000-MDT0001' with [0x200000005:0x2:0x0]: rc = 0 [ 5412.949440] Lustre: 268484:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0001: restore index '0x10000' with [0x200000003:0x4:0x0]: rc = 0 [ 5412.952605] Lustre: 268484:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0001: restore index '0x2010000' with [0x200000003:0x6:0x0]: rc = 0 [ 5412.955593] Lustre: 268484:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0001: restore index '0x2010000-MDT0001' with [0x200000005:0x3:0x0]: rc = 0 [ 5412.958826] Lustre: 268484:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0001: restore index '0x1010000' with [0x200000003:0x5:0x0]: rc = 0 [ 5412.965585] Lustre: Found index 1 for lustre-MDT0001, updating log [ 5413.033033] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5413.051782] Lustre: lustre-MDT0001: Aborting client recovery [ 5413.365107] loop2: detected capacity change from 0 to 204800000 [ 5413.404775] LDISKFS-fs (loop2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5413.437654] LDISKFS-fs (loop2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5413.459647] Lustre: 268570:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 5413.464185] Lustre: 268570:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0000: restore index '0x2020000-OST0000' with [0x200000005:0x3:0x0]: rc = 0 [ 5413.467336] Lustre: 268570:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0000: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 5413.471303] Lustre: 268570:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0000: restore index '0x20000-OST0000' with [0x200000005:0x1:0x0]: rc = 0 [ 5413.474846] Lustre: 268570:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0000: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 5413.477803] Lustre: 268570:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0000: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 5413.483233] Lustre: 268570:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0000: restore index '0x1020000-OST0000' with [0x200000005:0x2:0x0]: rc = 0 [ 5413.536792] Lustre: Found index 0 for lustre-OST0000, updating log [ 5413.574362] Lustre: lustre-OST0000: Aborting recovery [ 5413.876492] loop3: detected capacity change from 0 to 204800000 [ 5413.949078] LDISKFS-fs (loop3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5413.973072] LDISKFS-fs (loop3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5413.988485] Lustre: 268657:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 5413.993384] Lustre: 268657:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0001: restore index '0x1020000-OST0001' with [0x200000005:0x2:0x0]: rc = 0 [ 5413.998683] Lustre: 268657:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0001: restore index '0x2020000-OST0001' with [0x200000005:0x3:0x0]: rc = 0 [ 5414.004156] Lustre: 268657:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0001: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 5414.008955] Lustre: 268657:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0001: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 5414.013825] Lustre: 268657:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0001: restore index '0x20000-OST0001' with [0x200000005:0x1:0x0]: rc = 0 [ 5414.018312] Lustre: 268657:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0001: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 5414.054462] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 5414.056492] Lustre: Skipped 1 previous similar message [ 5419.495257] LustreError: 268427:0:(lod_dev.c:510:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 7, retries 0, failed: rc = -108 [ 5422.561740] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 5422.569517] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 5422.571690] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000bd0 [ 5422.579141] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000bd0 [ 5423.588052] Lustre: lustre-OST0001: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:5 to 0x2c0000400:33) [ 5423.588084] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:5 to 0x280000400:33) [ 5424.686207] LustreError: 268395:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0000: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0000 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0000/identity_upcall, time 214us: rc = -2 [ 5438.945869] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5438.946244] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5438.948766] Lustre: Skipped 3 previous similar messages [ 5438.950459] Lustre: Skipped 2 previous similar messages [ 5438.951510] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 5441.982237] LustreError: 269015:0:(obd_class.h:479:obd_check_dev()) Device 10 not setup [ 5441.984992] LustreError: 269015:0:(obd_class.h:479:obd_check_dev()) Skipped 1 previous similar message [ 5442.020495] Lustre: server umount lustre-MDT0000 complete [ 5442.308500] LustreError: 268377:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763086129 with bad export cookie 7540959768147666375 [ 5442.310240] LustreError: MGC192.168.204.154@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5442.312164] LustreError: 268377:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 5444.063631] LustreError: 268392:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5444.063859] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5444.064188] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 5444.069113] LustreError: 268392:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 5444.079558] Lustre: Skipped 1 previous similar message [ 5447.071459] LustreError: 268392:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5447.071561] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 5447.077339] LustreError: 268392:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 5447.080934] Lustre: Skipped 2 previous similar messages [ 5448.510165] LustreError: 269067:0:(obd_class.h:479:obd_check_dev()) Device 18 not setup [ 5448.512884] LustreError: 269067:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 5448.582197] Lustre: server umount lustre-MDT0001 complete [ 5455.021939] LustreError: 269119:0:(obd_class.h:479:obd_check_dev()) Device 23 not setup [ 5455.023932] LustreError: 269119:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 5455.052979] Lustre: server umount lustre-OST0000 complete [ 5461.419625] LustreError: 269172:0:(obd_class.h:479:obd_check_dev()) Device 2 not setup [ 5461.423045] LustreError: 269172:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 5461.473187] Lustre: server umount lustre-OST0001 complete [ 5464.248553] Lustre: DEBUG MARKER: == conf-sanity test 109a: test lctl clear_conf fsname ==== 21:09:10 (1763086150) [ 5473.642324] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing set_hostid [ 5476.808161] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing load_modules_local [ 5480.069341] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5482.095241] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5484.019151] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5486.110414] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5488.329828] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5488.348475] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5488.435892] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 5488.446212] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 5488.484754] Lustre: lustre-MDT0000: new disk, initializing [ 5488.508543] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5488.515356] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 5489.774334] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing set_default_debug -1 all [ 5493.846917] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5493.871606] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5493.907876] Lustre: 272713:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 5493.929323] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 5493.931508] Lustre: Skipped 1 previous similar message [ 5493.971819] Lustre: lustre-MDT0001: new disk, initializing [ 5494.000432] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5494.010393] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 5494.013735] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 5495.279807] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing set_default_debug -1 all [ 5498.245308] Lustre: DEBUG MARKER: oleg454-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 5499.402294] Lustre: DEBUG MARKER: oleg454-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 5501.580544] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5501.602978] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5501.702326] Lustre: lustre-OST0000: new disk, initializing [ 5501.705265] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 5503.520896] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing set_default_debug -1 all [ 5505.528173] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 5505.531466] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 5505.564815] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 5506.541085] Lustre: DEBUG MARKER: oleg454-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 5508.914986] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [ 5515.112202] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [ 5523.486325] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [ 5566.944923] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 5566.947230] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5566.948675] LustreError: Skipped 1 previous similar message [ 5566.955262] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5566.958309] Lustre: Skipped 1 previous similar message [ 5572.065599] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5572.068115] Lustre: Skipped 1 previous similar message [ 5580.255204] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 5580.274103] LustreError: 276420:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 5580.276955] LustreError: 276420:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 5580.317576] Lustre: server umount lustre-OST0000 complete [ 5582.304984] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5582.305824] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5582.310547] Lustre: Skipped 2 previous similar messages [ 5582.313481] Lustre: Skipped 2 previous similar messages [ 5587.744576] Lustre: server umount lustre-MDT0000 complete [ 5589.021135] LustreError: 272705:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763086276 with bad export cookie 7540959768147669084 [ 5589.024534] LustreError: MGC192.168.204.154@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5589.024763] LustreError: 272705:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 5589.143908] Lustre: server umount lustre-MDT0001 complete [ 5592.061549] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5593.374374] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing set_default_debug -1 all [ 5595.058920] Lustre: server umount MGS complete [ 5599.102311] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5599.268654] LustreError: 278387:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5599.277532] LustreError: 278399:0:(qmt_pool.c:1523:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [ 5599.301434] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5599.303717] Lustre: Skipped 1 previous similar message [ 5600.671571] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing set_default_debug -1 all [ 5603.754717] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5605.339808] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing set_default_debug -1 all [ 5607.336423] Lustre: DEBUG MARKER: oleg454-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 5608.643427] Lustre: DEBUG MARKER: oleg454-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 5611.142686] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5613.213771] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing set_default_debug -1 all [ 5615.117373] Lustre: DEBUG MARKER: oleg454-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 5645.282366] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5645.283248] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 5645.289576] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5645.291535] LustreError: Skipped 1 previous similar message [ 5645.293365] Lustre: Skipped 3 previous similar messages [ 5649.389677] LustreError: 281066:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 5649.391633] LustreError: 281066:0:(obd_class.h:479:obd_check_dev()) Skipped 19 previous similar messages [ 5649.429926] Lustre: server umount lustre-OST0000 complete [ 5655.007888] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 5655.011633] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5655.017550] Lustre: Skipped 1 previous similar message [ 5656.796923] Lustre: server umount lustre-MDT0000 complete [ 5658.051212] LustreError: 279107:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763086345 with bad export cookie 7540959768147670911 [ 5658.053196] LustreError: MGC192.168.204.154@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5658.054528] LustreError: 279107:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 1 previous similar message [ 5658.178815] Lustre: server umount lustre-MDT0001 complete [ 5661.963265] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing unload_modules_local [ 5663.033355] Key type lgssc unregistered [ 5663.175379] LNet: 281999:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5663.179301] LNetError: 281999:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5663.190307] LNet: Removed LNI 192.168.204.154@tcp [ 5663.485419] Key type .llcrypt unregistered [ 5663.486517] Key type ._llcrypt unregistered [ 5676.108806] Key type ._llcrypt registered [ 5676.109763] Key type .llcrypt registered [ 5676.148394] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing set_hostid [ 5682.615814] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing load_modules_local [ 5682.928168] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5682.934075] alg: No test for adler32 (adler32-zlib) [ 5683.808232] Lustre: Lustre: Build Version: 2.16.61_1_gaa570bc [ 5683.901152] LNet: Added LNI 192.168.204.154@tcp [8/256/0/180] [ 5685.487193] Key type lgssc registered [ 5685.865758] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5688.997366] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5691.278766] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5693.456940] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5695.775932] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5698.309154] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5698.328982] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5698.336378] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5699.439539] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 5699.456511] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 5699.499723] Lustre: lustre-MDT0000: new disk, initializing [ 5699.525058] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5699.532216] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 5701.086308] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing set_default_debug -1 all [ 5705.174434] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5705.196568] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5705.229033] Lustre: 285902:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 5705.252427] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 5705.253943] Lustre: Skipped 1 previous similar message [ 5705.288858] Lustre: lustre-MDT0001: new disk, initializing [ 5705.321551] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5705.332818] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 5705.337407] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 5706.591351] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing set_default_debug -1 all [ 5709.627384] Lustre: DEBUG MARKER: oleg454-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 5710.835441] Lustre: DEBUG MARKER: oleg454-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 5713.064602] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5713.087559] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5713.184033] Lustre: lustre-OST0000: new disk, initializing [ 5713.186760] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 5713.215477] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5713.573884] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 5713.577964] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 5713.619586] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 5715.056944] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing set_default_debug -1 all [ 5718.163072] Lustre: DEBUG MARKER: oleg454-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 5721.594974] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [ 5724.049111] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [ 5732.589378] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [ 5775.331645] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5775.334304] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 5775.335394] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5775.335589] Lustre: Skipped 1 previous similar message [ 5775.341258] LustreError: Skipped 1 previous similar message [ 5778.030511] LustreError: 289461:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 5778.069302] Lustre: server umount lustre-OST0000 complete [ 5780.450398] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5780.451470] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5780.454282] Lustre: Skipped 1 previous similar message [ 5780.455989] Lustre: Skipped 1 previous similar message [ 5785.540493] LustreError: 289663:0:(obd_class.h:479:obd_check_dev()) Device 22 not setup [ 5785.543253] LustreError: 289663:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 5785.583883] Lustre: server umount lustre-MDT0000 complete [ 5786.892197] LustreError: 287648:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763086474 with bad export cookie 12506701879086243625 [ 5786.895931] LustreError: MGC192.168.204.154@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5786.897879] LustreError: 287648:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 5786.979049] LustreError: 289864:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 5786.981222] LustreError: 289864:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 5787.046611] Lustre: server umount lustre-MDT0001 complete [ 5790.062185] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5791.420395] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing set_default_debug -1 all [ 5792.994543] LustreError: 290804:0:(obd_class.h:479:obd_check_dev()) Device 2 not setup [ 5792.996608] LustreError: 290804:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 5793.016633] Lustre: server umount MGS complete [ 5796.817384] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5796.994935] LustreError: 291427:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5797.001038] LustreError: 291427:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 5797.003222] LustreError: 291439:0:(qmt_pool.c:1523:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [ 5797.028362] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5798.448772] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing set_default_debug -1 all [ 5801.470372] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5801.607282] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5803.048624] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing set_default_debug -1 all [ 5804.945154] Lustre: DEBUG MARKER: oleg454-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 5806.200806] Lustre: DEBUG MARKER: oleg454-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 5808.675838] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5810.766860] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing set_default_debug -1 all [ 5812.564195] Lustre: DEBUG MARKER: oleg454-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 5837.793308] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5837.796529] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 5837.797702] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5837.800039] LustreError: Skipped 1 previous similar message [ 5837.802312] Lustre: Skipped 2 previous similar messages [ 5842.799072] LustreError: 294106:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 5842.800978] LustreError: 294106:0:(obd_class.h:479:obd_check_dev()) Skipped 1 previous similar message [ 5842.836415] Lustre: server umount lustre-OST0000 complete [ 5848.032925] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 5848.032976] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5848.033433] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5848.039827] Lustre: Skipped 2 previous similar messages [ 5850.267530] Lustre: server umount lustre-MDT0000 complete [ 5851.520286] LustreError: 291454:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763086538 with bad export cookie 12506701879086245452 [ 5851.523528] LustreError: MGC192.168.204.154@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5851.524417] LustreError: 291454:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 5851.585413] LustreError: 294508:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [ 5851.587025] LustreError: 294508:0:(obd_class.h:479:obd_check_dev()) Skipped 9 previous similar messages [ 5851.636444] Lustre: server umount lustre-MDT0001 complete [ 5855.523586] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing unload_modules_local [ 5856.562324] Key type lgssc unregistered [ 5856.698507] LNet: 295039:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5856.702448] LNetError: 295039:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5856.712389] LNet: Removed LNI 192.168.204.154@tcp [ 5857.036258] Key type .llcrypt unregistered [ 5857.037939] Key type ._llcrypt unregistered [ 5864.154460] Key type ._llcrypt registered [ 5864.155460] Key type .llcrypt registered [ 5864.190515] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing set_hostid [ 5870.496020] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing load_modules_local [ 5870.846447] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5870.858649] alg: No test for adler32 (adler32-zlib) [ 5871.722782] Lustre: Lustre: Build Version: 2.16.61_1_gaa570bc [ 5871.809944] LNet: Added LNI 192.168.204.154@tcp [8/256/0/180] [ 5873.399189] Key type lgssc registered [ 5873.772487] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5876.470993] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5878.342733] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5881.055216] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing load_modules_local [ 5883.672415] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5885.813317] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5885.835169] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5885.841881] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5886.929699] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 5886.940600] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 5886.975730] Lustre: lustre-MDT0000: new disk, initializing [ 5887.000490] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5887.008393] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 5888.283801] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing set_default_debug -1 all [ 5892.384217] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5892.410490] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5892.524903] Lustre: lustre-OST0000: new disk, initializing [ 5892.525848] Lustre: lustre-OST0000: Not available for connect from 0@lo (not set up) [ 5892.527917] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 5892.532074] Lustre: Skipped 1 previous similar message [ 5892.554259] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5894.370938] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing set_default_debug -1 all [ 5897.712219] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 5897.714672] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 5897.723343] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 6076.896972] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6076.901864] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6078.532742] LustreError: 299952:0:(obd_class.h:479:obd_check_dev()) Device 11 not setup [ 6078.644586] Lustre: server umount lustre-MDT0000 complete [ 6079.989506] LustreError: 299954:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763086767 with bad export cookie 16419848231848777820 [ 6079.991778] LustreError: MGC192.168.204.154@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6079.993465] LustreError: 299954:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 6080.018172] LustreError: 300156:0:(obd_class.h:479:obd_check_dev()) Device 2 not setup [ 6080.020817] LustreError: 300156:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 6080.068412] Lustre: server umount lustre-OST0000 complete [ 6088.646731] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing set_hostid [ 6091.659109] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing load_modules_local [ 6094.959067] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6097.166353] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6099.151558] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6101.294141] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6103.500529] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6103.522265] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6103.604774] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 6103.615521] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 6103.656947] Lustre: lustre-MDT0000: new disk, initializing [ 6103.682872] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6103.689917] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 6104.951542] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing set_default_debug -1 all [ 6109.066549] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6109.090424] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 6109.123013] Lustre: 303192:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 6109.147166] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 6109.149803] Lustre: Skipped 1 previous similar message [ 6109.189755] Lustre: lustre-MDT0001: new disk, initializing [ 6109.223744] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 6109.237350] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 6109.240300] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 6110.558909] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing set_default_debug -1 all [ 6113.592543] Lustre: DEBUG MARKER: oleg454-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 6114.772993] Lustre: DEBUG MARKER: oleg454-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 6116.990415] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6117.014029] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 6117.103707] Lustre: lustre-OST0000: new disk, initializing [ 6117.106520] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 6117.141495] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 6118.919020] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing set_default_debug -1 all [ 6121.210401] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 6121.213625] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 6121.251353] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 6121.925960] Lustre: DEBUG MARKER: oleg454-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 6123.287407] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 6123.361970] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 6124.643533] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 6124.717773] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 6126.564092] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6126.564134] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 6126.568518] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6126.572354] LustreError: Skipped 1 previous similar message [ 6126.576144] Lustre: Skipped 1 previous similar message [ 6127.405859] LustreError: 306127:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 6127.407442] LustreError: 306127:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 6127.445351] Lustre: server umount lustre-OST0000 complete [ 6129.631796] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 6129.633851] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6129.636930] Lustre: Skipped 1 previous similar message [ 6129.638424] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6131.681259] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6134.784601] LustreError: 306329:0:(obd_class.h:479:obd_check_dev()) Device 22 not setup [ 6134.786580] LustreError: 306329:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 6134.820407] Lustre: server umount lustre-MDT0000 complete [ 6136.067185] LustreError: 303972:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763086823 with bad export cookie 16419848231851718611 [ 6136.073048] LustreError: MGC192.168.204.154@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6136.193647] Lustre: server umount lustre-MDT0001 complete [ 6137.328927] Lustre: DEBUG MARKER: SKIP: conf-sanity test_111 skipping SLOW test 111 [ 6137.922359] Lustre: DEBUG MARKER: == conf-sanity test 112a: mount OST with no_create option ========================================================== 21:20:24 (1763086824) [ 6140.206223] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6140.378875] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6141.660345] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing set_default_debug -1 all [ 6144.576704] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 6145.985472] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing set_default_debug -1 all [ 6147.764041] Lustre: DEBUG MARKER: oleg454-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 6148.927957] Lustre: DEBUG MARKER: oleg454-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 6151.327642] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 6153.474793] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing set_default_debug -1 all [ 6155.349821] Lustre: DEBUG MARKER: oleg454-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 6157.729353] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6157.752770] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 6157.803032] Lustre: lustre-OST0001: new disk, initializing [ 6157.804993] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 6157.831993] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 6157.833992] Lustre: Skipped 2 previous similar messages [ 6158.603188] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 6158.607668] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 6158.624224] LustreError: 309737:0:(osp_precreate.c:969:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [ 6159.651484] LustreError: 309749:0:(osp_precreate.c:969:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [ 6159.654531] LustreError: 309749:0:(osp_precreate.c:969:osp_precreate_cleanup_orphans()) Skipped 2 previous similar messages [ 6159.755950] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing set_default_debug -1 all [ 6160.674665] LustreError: 309737:0:(osp_precreate.c:969:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [ 6160.679638] LustreError: 309737:0:(osp_precreate.c:969:osp_precreate_cleanup_orphans()) Skipped 1 previous similar message [ 6162.722902] LustreError: 309737:0:(osp_precreate.c:969:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [ 6162.727174] LustreError: 309737:0:(osp_precreate.c:969:osp_precreate_cleanup_orphans()) Skipped 3 previous similar messages [ 6163.652907] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 6165.794436] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 2 sec [ 6166.820581] LustreError: 309737:0:(osp_precreate.c:969:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [ 6166.826292] LustreError: 309737:0:(osp_precreate.c:969:osp_precreate_cleanup_orphans()) Skipped 7 previous similar messages [ 6167.095094] Lustre: DEBUG MARKER: oleg454-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff9587cd63b800.ost_server_uuid 50 [ 6167.668614] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff9587cd63b800.ost_server_uuid in FULL state after 0 sec [ 6168.995839] Lustre: DEBUG MARKER: oleg454-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff9587cd63b800.ost_server_uuid 50 [ 6169.507968] Lustre: DEBUG MARKER: osc.lustre-OST0001-osc-ffff9587cd63b800.ost_server_uuid in FULL state after 0 sec [ 6170.920158] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 6181.344436] LustreError: lustre-OST0001-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 6181.344541] Lustre: lustre-OST0001-osc-MDT0001: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6181.346886] LustreError: Skipped 1 previous similar message [ 6181.350597] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 6181.353646] Lustre: Skipped 3 previous similar messages [ 6187.310106] LustreError: 310921:0:(obd_class.h:479:obd_check_dev()) Device 25 not setup [ 6187.312689] LustreError: 310921:0:(obd_class.h:479:obd_check_dev()) Skipped 13 previous similar messages [ 6187.330798] Lustre: server umount lustre-OST0001 complete [ 6188.098190] LustreError: 309738:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 192.168.204.54@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6188.103560] LustreError: 309738:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 6190.564125] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6190.567425] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 6190.568175] Lustre: Skipped 2 previous similar messages [ 6190.568276] LustreError: 308696:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6190.568294] LustreError: 308696:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 6190.568744] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6190.568752] Lustre: Skipped 3 previous similar messages [ 6195.156115] Lustre: server umount lustre-OST0000 complete [ 6202.560754] LustreError: 311323:0:(obd_class.h:479:obd_check_dev()) Device 28 not setup [ 6202.563034] LustreError: 311323:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 6202.589210] Lustre: server umount lustre-MDT0000 complete [ 6203.962872] LustreError: 307081:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763086891 with bad export cookie 16419848231851719479 [ 6203.966195] LustreError: 307081:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 5 previous similar messages [ 6203.967127] LustreError: MGC192.168.204.154@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6204.102669] Lustre: server umount lustre-MDT0001 complete [ 6208.061754] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing unload_modules_local [ 6209.039812] Key type lgssc unregistered [ 6209.173284] LNet: 312056:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 6209.175571] LNetError: 312056:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 6209.184285] LNet: Removed LNI 192.168.204.154@tcp [ 6209.477498] Key type .llcrypt unregistered [ 6209.479144] Key type ._llcrypt unregistered [ 6218.554621] Key type ._llcrypt registered [ 6218.555693] Key type .llcrypt registered [ 6218.592075] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing load_modules_local [ 6218.890160] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 6218.928983] alg: No test for adler32 (adler32-zlib) [ 6219.776859] Lustre: Lustre: Build Version: 2.16.61_1_gaa570bc [ 6219.853813] LNet: Added LNI 192.168.204.154@tcp [8/256/0/180] [ 6221.431176] Key type lgssc registered [ 6221.800056] Lustre: Echo OBD driver; http://www.lustre.org/ [ 6224.491309] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 6224.497701] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6225.681180] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6226.914890] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing set_default_debug -1 all [ 6228.726511] Lustre: DEBUG MARKER: oleg454-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 6231.039059] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 6231.169737] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 6232.530350] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing set_default_debug -1 all [ 6235.829636] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 6235.959605] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 6236.963159] LustreError: 314987:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6236.969236] LustreError: 314987:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 6236.972360] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:65) [ 6237.964038] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing set_default_debug -1 all [ 6239.868842] Lustre: DEBUG MARKER: oleg454-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 6241.057576] LustreError: 315169:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6241.064896] LustreError: 315169:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 6242.118200] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 6242.182729] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 6243.901333] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing set_default_debug -1 all [ 6244.857410] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:33) [ 6245.672072] Lustre: DEBUG MARKER: oleg454-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid [ 6248.061105] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing wait_import_state (FULL|IDLE) os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [ 6248.124820] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 6326.752573] LustreError: lustre-OST0001-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 6326.752648] Lustre: lustre-OST0001-osc-MDT0001: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6326.755950] LustreError: Skipped 1 previous similar message [ 6326.760815] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 6326.764759] Lustre: Skipped 1 previous similar message [ 6327.265900] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 6327.267898] Lustre: Skipped 1 previous similar message [ 6328.685745] LustreError: 316856:0:(obd_class.h:479:obd_check_dev()) Device 27 not setup [ 6328.702539] Lustre: server umount lustre-OST0001 complete [ 6332.385478] LustreError: 314988:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6332.386343] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6332.388504] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6332.391443] LustreError: 314988:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 6332.396765] Lustre: Skipped 2 previous similar messages [ 6336.299057] LustreError: 317059:0:(obd_class.h:479:obd_check_dev()) Device 23 not setup [ 6336.300764] LustreError: 317059:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 6336.335906] Lustre: server umount lustre-OST0000 complete [ 6339.040274] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 6339.042478] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6339.047492] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6339.050480] Lustre: Skipped 1 previous similar message [ 6339.646974] LustreError: 317260:0:(obd_class.h:479:obd_check_dev()) Device 10 not setup [ 6339.649669] LustreError: 317260:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 6339.676323] Lustre: server umount lustre-MDT0000 complete [ 6340.850093] LustreError: 314254:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763087028 with bad export cookie 11003306440062339705 [ 6340.853525] LustreError: MGC192.168.204.154@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6340.853694] LustreError: 314254:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 6340.967753] Lustre: server umount lustre-MDT0001 complete [ 6344.312176] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing unload_modules_local [ 6345.305881] Key type lgssc unregistered [ 6345.420542] LNet: 317992:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 6345.422646] LNetError: 317992:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 6345.431260] LNet: Removed LNI 192.168.204.154@tcp [ 6345.707828] Key type .llcrypt unregistered [ 6345.709078] Key type ._llcrypt unregistered [ 6362.729141] Key type ._llcrypt registered [ 6362.730611] Key type .llcrypt registered [ 6362.766834] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing load_modules_local [ 6363.156694] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 6363.197469] alg: No test for adler32 (adler32-zlib) [ 6364.056089] Lustre: Lustre: Build Version: 2.16.61_1_gaa570bc [ 6364.137254] LNet: Added LNI 192.168.204.154@tcp [8/256/0/180] [ 6365.719203] Key type lgssc registered [ 6366.084429] Lustre: Echo OBD driver; http://www.lustre.org/ [ 6369.229906] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 6369.235288] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6370.375301] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6371.553304] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing set_default_debug -1 all [ 6374.256373] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 6374.377322] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 6375.608624] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing set_default_debug -1 all [ 6376.549864] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 6378.681762] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 6378.803735] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 6379.810304] LustreError: 321694:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6379.813884] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:97) [ 6379.816148] LustreError: 321694:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 6380.736331] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing set_default_debug -1 all [ 6383.551922] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 6383.634769] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 6384.682939] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:65) [ 6385.414625] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing set_default_debug -1 all [ 6389.462439] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 6395.089357] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 6405.090847] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6405.091919] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6405.095124] Lustre: Skipped 3 previous similar messages [ 6405.095530] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 6405.096840] Lustre: Skipped 3 previous similar messages [ 6409.595439] LustreError: 323648:0:(obd_class.h:479:obd_check_dev()) Device 10 not setup [ 6409.618735] Lustre: server umount lustre-MDT0000 complete [ 6410.211279] LustreError: 322498:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6410.218230] LustreError: 322498:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 6410.780200] LustreError: 320216:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763087098 with bad export cookie 15150584889550719181 [ 6410.783527] LustreError: MGC192.168.204.154@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6410.785551] LustreError: 320216:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 6410.842358] LustreError: 323849:0:(obd_class.h:479:obd_check_dev()) Device 18 not setup [ 6410.845020] LustreError: 323849:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 6410.894610] Lustre: server umount lustre-MDT0001 complete [ 6422.512133] LustreError: 324049:0:(obd_class.h:479:obd_check_dev()) Device 23 not setup [ 6422.513571] LustreError: 324049:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 6422.530121] Lustre: server umount lustre-OST0000 complete [ 6433.904660] LustreError: 324251:0:(obd_class.h:479:obd_check_dev()) Device 2 not setup [ 6433.906829] LustreError: 324251:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 6433.945130] Lustre: server umount lustre-OST0001 complete [ 6439.011987] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing load_modules_local [ 6442.302563] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6442.435222] LustreError: 325426:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6442.454818] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6443.730672] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing set_default_debug -1 all [ 6446.621496] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 6448.062373] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing set_default_debug -1 all [ 6448.980922] Lustre: 326534:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 6451.114886] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 6451.251551] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 6451.254245] Lustre: Skipped 1 previous similar message [ 6452.262515] LustreError: 326890:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6452.265854] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:129) [ 6453.030403] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing set_default_debug -1 all [ 6455.837626] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 6456.944223] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:97) [ 6457.828390] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing set_default_debug -1 all [ 6460.872379] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 6462.039741] Lustre: 328381:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 6472.673584] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6472.676373] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 6472.676785] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6472.677503] Lustre: Skipped 2 previous similar messages [ 6474.175116] LustreError: 328486:0:(obd_class.h:479:obd_check_dev()) Device 10 not setup [ 6474.177243] LustreError: 328486:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 6474.202816] Lustre: server umount lustre-MDT0000 complete [ 6475.374725] LustreError: 325407:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763087162 with bad export cookie 15150584889550720833 [ 6475.378059] LustreError: 325407:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 6475.378661] LustreError: MGC192.168.204.154@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6475.499091] Lustre: server umount lustre-MDT0001 complete [ 6487.027232] LustreError: 328889:0:(obd_class.h:479:obd_check_dev()) Device 23 not setup [ 6487.029848] LustreError: 328889:0:(obd_class.h:479:obd_check_dev()) Skipped 15 previous similar messages [ 6487.046777] Lustre: server umount lustre-OST0000 complete [ 6498.533843] Lustre: server umount lustre-OST0001 complete [ 6499.800402] Lustre: DEBUG MARKER: SKIP: conf-sanity test_114 skipping SLOW test 114 [ 6500.391314] Lustre: DEBUG MARKER: SKIP: conf-sanity test_115 skipping excluded test 115 [ 6500.994644] Lustre: DEBUG MARKER: == conf-sanity test 116: big size MDT support ============ 21:26:27 (1763087187) [ 6508.895317] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing load_modules_local [ 6512.089365] loop0: detected capacity change from 0 to 1099511627776 [ 6512.254429] SGI XFS with ACLs, security attributes, quota, no debug enabled [ 6512.257956] *** VALIDATE xfs *** [ 6512.263667] XFS (loop0): Mounting V5 Filesystem [ 6512.274370] XFS (loop0): Ending clean mount [ 6512.728714] loop1: detected capacity change from 0 to 18691697672192 [ 6528.968801] LDISKFS-fs (loop1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6534.964983] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing set_hostid [ 6538.261184] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing load_modules_local [ 6541.801794] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6544.030569] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6546.250327] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6548.497077] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6551.072192] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6551.104530] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6551.219375] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 6551.234829] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 6551.279784] Lustre: lustre-MDT0000: new disk, initializing [ 6551.307783] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6551.310640] Lustre: Skipped 1 previous similar message [ 6551.318062] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 6552.719758] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing set_default_debug -1 all [ 6557.048154] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6557.073928] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 6557.113556] Lustre: 333459:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 6557.137813] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 6557.140066] Lustre: Skipped 1 previous similar message [ 6557.176531] Lustre: lustre-MDT0001: new disk, initializing [ 6557.234208] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 6557.238933] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 6558.690482] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing set_default_debug -1 all [ 6561.934685] Lustre: DEBUG MARKER: oleg454-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 6563.211264] Lustre: DEBUG MARKER: oleg454-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 6565.624301] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6565.648764] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 6565.768343] Lustre: lustre-OST0000: new disk, initializing [ 6565.770548] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 6567.356635] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 6567.359777] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 6567.402197] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 6567.812528] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing set_default_debug -1 all [ 6571.079824] Lustre: DEBUG MARKER: oleg454-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 6572.584578] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 6572.676054] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 6574.145718] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 6574.234919] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 6577.634346] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 6577.635142] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6577.636871] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6577.636875] Lustre: Skipped 3 previous similar messages [ 6577.639350] LustreError: Skipped 1 previous similar message [ 6577.643589] Lustre: Skipped 1 previous similar message [ 6582.755230] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6582.757866] Lustre: Skipped 2 previous similar messages [ 6587.877939] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6587.879802] Lustre: Skipped 1 previous similar message [ 6589.407122] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 6589.429087] LustreError: 336400:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 6589.431184] LustreError: 336400:0:(obd_class.h:479:obd_check_dev()) Skipped 9 previous similar messages [ 6589.474665] Lustre: server umount lustre-OST0000 complete [ 6592.993147] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6592.993411] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 6592.997658] Lustre: Skipped 1 previous similar message [ 6598.112284] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6598.114635] Lustre: Skipped 3 previous similar messages [ 6605.279212] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 6605.351455] Lustre: server umount lustre-MDT0000 complete [ 6606.619080] LustreError: 333449:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763087294 with bad export cookie 15150584889550722394 [ 6606.622341] LustreError: MGC192.168.204.154@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6606.622484] LustreError: 333449:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 6606.745209] Lustre: server umount lustre-MDT0001 complete [ 6609.607099] XFS (loop0): Unmounting Filesystem [ 6611.224456] Lustre: DEBUG MARKER: == conf-sanity test 117: lctl get_param return errors properly ========================================================== 21:28:17 (1763087297) [ 6613.608046] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6613.771386] LustreError: 337620:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6613.799496] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6613.801666] Lustre: Skipped 2 previous similar messages [ 6615.109824] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing set_default_debug -1 all [ 6617.976062] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 6619.329251] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing set_default_debug -1 all [ 6621.174619] Lustre: DEBUG MARKER: oleg454-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 6622.361504] Lustre: DEBUG MARKER: oleg454-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 6624.671838] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 6626.650425] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing set_default_debug -1 all [ 6628.491936] Lustre: DEBUG MARKER: oleg454-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 6643.167880] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 6643.170628] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6643.175094] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6643.176972] Lustre: Skipped 2 previous similar messages [ 6648.365432] LustreError: 339981:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 6648.367439] LustreError: 339981:0:(obd_class.h:479:obd_check_dev()) Skipped 17 previous similar messages [ 6648.405562] Lustre: server umount lustre-OST0000 complete [ 6653.919902] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 6653.922764] LustreError: Skipped 1 previous similar message [ 6653.924062] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6653.927803] Lustre: Skipped 1 previous similar message [ 6655.829916] Lustre: server umount lustre-MDT0000 complete [ 6657.113530] LustreError: 337601:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763087344 with bad export cookie 15150584889550723276 [ 6657.119649] LustreError: MGC192.168.204.154@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6657.245647] Lustre: server umount lustre-MDT0001 complete [ 6660.691417] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing unload_modules_local [ 6661.599379] Key type lgssc unregistered [ 6661.714369] LNet: 340917:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 6661.716487] LNetError: 340917:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 6661.725283] LNet: Removed LNI 192.168.204.154@tcp [ 6662.005100] Key type .llcrypt unregistered [ 6662.006118] Key type ._llcrypt unregistered [ 6672.830706] Key type ._llcrypt registered [ 6672.831739] Key type .llcrypt registered [ 6672.866771] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing load_modules_local [ 6673.215345] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 6673.244717] alg: No test for adler32 (adler32-zlib) [ 6674.094649] Lustre: Lustre: Build Version: 2.16.61_1_gaa570bc [ 6674.189626] LNet: Added LNI 192.168.204.154@tcp [8/256/0/180] [ 6675.775196] Key type lgssc registered [ 6676.128372] Lustre: Echo OBD driver; http://www.lustre.org/ [ 6678.744513] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 6678.750044] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6679.908881] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6681.132122] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing set_default_debug -1 all [ 6683.999520] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 6684.140248] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 6685.513108] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing set_default_debug -1 all [ 6687.309409] Lustre: DEBUG MARKER: oleg454-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 6688.419191] Lustre: DEBUG MARKER: oleg454-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 6690.872202] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 6690.981713] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 6693.241621] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing set_default_debug -1 all [ 6695.037180] Lustre: DEBUG MARKER: oleg454-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 6709.727905] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [ 6709.731737] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6709.737966] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 6710.753721] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6710.753901] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 6710.759526] Lustre: Skipped 1 previous similar message [ 6714.303394] LustreError: 344754:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [ 6714.326833] Lustre: server umount lustre-MDT0001 complete [ 6715.273142] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6715.875109] LustreError: 342437:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6715.880098] LustreError: 342437:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 6717.882698] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 6717.914434] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [ 6717.917508] Lustre: Found index 1 for lustre-MDT0001, updating log [ 6717.929368] Lustre: 342430:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 6718.054512] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 6719.317223] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing set_default_debug -1 all [ 6723.557168] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 6725.093378] LustreError: 345884:0:(obd_mount.c:188:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [ 6725.096298] LustreError: 345884:0:(tgt_mount.c:669:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [ 6725.099717] LustreError: 342433:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [ 6725.117739] LustreError: 345887:0:(obd_config.c:1997:class_config_llog_handler()) MGC192.168.204.154@tcp: cfg command failed: rc = -17 [ 6725.120605] Lustre: cmd=cf001 0:lustre-MDT0001-osp-MDT0000 1:osp 2:lustre-MDT0000-mdtlov_UUID [ 6725.120605] [ 7023.175677] Lustre: lustre-MDT0001: Not available for connect from 192.168.204.54@tcp (stopping) [ 7023.179458] Lustre: Skipped 1 previous similar message [ 7025.634983] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7025.635357] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [ 7025.637471] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 7025.639485] Lustre: Skipped 1 previous similar message [ 7028.734266] LustreError: 357225:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [ 7028.736798] LustreError: 357225:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 7028.760643] Lustre: server umount lustre-MDT0001 complete [ 7029.760908] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 7030.752530] LustreError: 346392:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7030.757108] LustreError: 346392:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 7032.266084] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 7032.306943] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [ 7032.310447] Lustre: Found index 1 for lustre-MDT0001, updating log [ 7032.324795] Lustre: 342430:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 7032.455823] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7033.709858] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing set_default_debug -1 all [ 7037.926615] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 7039.461696] LustreError: 358351:0:(obd_mount.c:188:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [ 7039.464234] LustreError: 358351:0:(tgt_mount.c:669:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [ 7039.467025] LustreError: 342433:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [ 7039.469865] LustreError: 342433:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [ 7039.486378] LustreError: 358354:0:(obd_config.c:689:class_setup()) Device 8 already setup (type osp) [ 7039.488827] LustreError: 358354:0:(obd_config.c:1997:class_config_llog_handler()) MGC192.168.204.154@tcp: cfg command failed: rc = -17 [ 7039.491616] Lustre: cmd=cf003 0:lustre-MDT0001-osp-MDT0000 1:lustre-MDT0001_UUID 2:192.168.204.154@tcp [ 7039.491616] [ 7340.001739] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7340.004858] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [ 7340.005258] Lustre: Skipped 1 previous similar message [ 7340.005992] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 7340.006032] Lustre: Skipped 2 previous similar messages [ 7350.239126] Lustre: lustre-MDT0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 7350.240839] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 7350.245455] Lustre: Skipped 6 previous similar messages [ 7350.278326] LustreError: 369741:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [ 7350.280218] LustreError: 369741:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 7350.304428] Lustre: server umount lustre-MDT0001 complete [ 7350.848625] LustreError: 342435:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.204.54@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7351.275617] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 7353.861766] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 7353.895262] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [ 7353.897557] Lustre: Found index 1 for lustre-MDT0001, updating log [ 7353.912089] Lustre: 342430:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 7354.035505] Lustre: lustre-MDT0001: Imperative Recovery enabled, recovery window shrunk from 60-180 down to 60-180 [ 7355.377354] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing set_default_debug -1 all [ 7359.465827] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 7363.046310] LustreError: 370921:0:(obd_mount.c:188:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [ 7363.049215] LustreError: 370921:0:(tgt_mount.c:669:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [ 7363.052461] LustreError: 342433:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [ 7363.055552] LustreError: 342433:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [ 7363.076945] LustreError: 370928:0:(lod_lov.c:207:lod_add_device()) lustre-MDT0001-osp-MDT0000: cannot connect to next dev lustre-MDT0001_UUID (-114) [ 7363.085469] LustreError: 370928:0:(obd_config.c:1997:class_config_llog_handler()) MGC192.168.204.154@tcp: cfg command failed: rc = -114 [ 7363.090920] Lustre: cmd=cf014 0:lustre-MDT0000-mdtlov 1:lustre-MDT0001_UUID 2:1 3:1 [ 7363.090920] [ 7671.777929] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7671.778670] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7671.780657] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7671.784438] Lustre: Skipped 1 previous similar message [ 7671.787499] Lustre: Skipped 2 previous similar messages [ 7675.840098] LustreError: 382425:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup [ 7675.841885] LustreError: 382425:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 7675.876863] Lustre: server umount lustre-MDT0000 complete [ 7676.897510] LustreError: 342436:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7676.902128] LustreError: 342436:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 7677.109124] LustreError: 342423:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763088364 with bad export cookie 18378884868513978821 [ 7677.111993] LustreError: MGC192.168.204.154@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7677.115021] LustreError: 342423:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 7682.015918] LustreError: 342436:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7682.016267] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7682.024116] Lustre: Skipped 1 previous similar message [ 7683.265256] LustreError: 382629:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [ 7683.267756] LustreError: 382629:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 7683.327755] Lustre: server umount lustre-MDT0001 complete [ 7684.663456] Lustre: server umount lustre-OST0000 complete [ 7687.622747] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing set_hostid [ 7690.687312] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing load_modules_local [ 7694.131767] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 7696.209048] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 7698.304228] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 7700.545576] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 7702.898941] Lustre: DEBUG MARKER: == conf-sanity test 120: cross-target rename should not create bad symlinks ========================================================== 21:46:29 (1763088389) [ 7705.181847] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 7705.210095] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7705.295887] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 7705.306114] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 7705.344556] Lustre: lustre-MDT0000: new disk, initializing [ 7705.368225] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7705.374333] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 7706.604637] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing set_default_debug -1 all [ 7710.675284] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 7710.697736] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 7710.726775] Lustre: 385357:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 7710.747836] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 7710.749501] Lustre: Skipped 1 previous similar message [ 7710.796871] Lustre: lustre-MDT0001: new disk, initializing [ 7710.828876] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7710.841022] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 7710.843973] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 7712.062950] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing set_default_debug -1 all [ 7715.117100] Lustre: DEBUG MARKER: oleg454-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 7716.316938] Lustre: DEBUG MARKER: oleg454-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 7718.610666] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 7718.633501] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 7718.746910] Lustre: lustre-OST0000: new disk, initializing [ 7718.749601] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 7718.780282] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7720.615879] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing set_default_debug -1 all [ 7723.717520] Lustre: DEBUG MARKER: oleg454-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 7726.591410] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 7726.595120] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 7726.635905] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 7736.801938] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7736.802154] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7736.808656] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7736.810129] Lustre: Skipped 3 previous similar messages [ 7737.854535] LustreError: 387805:0:(obd_class.h:479:obd_check_dev()) Device 22 not setup [ 7737.856386] LustreError: 387805:0:(obd_class.h:479:obd_check_dev()) Skipped 11 previous similar messages [ 7737.887638] Lustre: server umount lustre-MDT0000 complete [ 7739.122095] LustreError: 386138:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1763088426 with bad export cookie 18378884868513980774 [ 7739.124337] LustreError: 386151:0:(osp_object.c:617:osp_attr_get()) lustre-MDT0000-osp-MDT0001: osp_attr_get update error [0x200000401:0x2:0x0]: rc = -5 [ 7739.125310] LustreError: 386138:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 1 previous similar message [ 7739.125877] LustreError: MGC192.168.204.154@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7739.246317] Lustre: server umount lustre-MDT0001 complete [ 7749.791090] Lustre: server umount lustre-OST0000 complete [ 7759.303690] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing set_hostid [ 7762.597501] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing load_modules_local [ 7766.124574] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 7768.372938] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 7770.544207] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 7772.830931] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 7773.431590] Lustre: DEBUG MARKER: == conf-sanity test complete, duration 7525 sec ========== 21:47:39 (1763088459) [ 7773.970663] Lustre: DEBUG MARKER: === conf-sanity: start cleanup 21:47:40 (1763088460) === [ 7775.035648] Lustre: DEBUG MARKER: === conf-sanity: finish cleanup 21:47:41 (1763088461) === [ 7788.580242] Lustre: DEBUG MARKER: oleg454-server.virtnet: executing unload_modules_local [ 7789.620269] Key type lgssc unregistered [ 7789.748385] LNet: 393310:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 7789.751864] LNetError: 393310:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 7789.761454] LNet: Removed LNI 192.168.204.154@tcp [ 7790.087105] Key type .llcrypt unregistered [ 7790.088915] Key type ._llcrypt unregistered