[ 0.000000] Linux version 4.18.0rh8.10-debug (green@maintenance) (gcc version 8.5.0 20210514 (Red Hat 8.5.0-26) (GCC)) #2 SMP Mon Jul 14 01:24:22 EDT 2025 [ 0.000000] Command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers' [ 0.000000] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256 [ 0.000000] x86/fpu: Enabled xstate features 0x7, context size is 832 bytes, using 'standard' format. [ 0.000000] signal: max sigframe size: 1776 [ 0.000000] BIOS-provided physical RAM map: [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x00000000bffcdfff] usable [ 0.000000] BIOS-e820: [mem 0x00000000bffce000-0x00000000bfffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x0000000146dfffff] usable [ 0.000000] NX (Execute Disable) protection: active [ 0.000000] SMBIOS 2.8 present. [ 0.000000] DMI: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.16.2-1.fc38 04/01/2014 [ 0.000000] Hypervisor detected: KVM [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00 [ 0.000000] kvm-clock: using sched offset of 530221214 cycles [ 0.000000] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns [ 0.000000] tsc: Detected 2399.998 MHz processor [ 0.000000] last_pfn = 0x146e00 max_arch_pfn = 0x400000000 [ 0.000000] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT [ 0.000000] last_pfn = 0xbffce max_arch_pfn = 0x400000000 [ 0.000000] found SMP MP-table at [mem 0x000f5b30-0x000f5b3f] [ 0.000000] RAMDISK: [mem 0xbcc54000-0xbffbffff] [ 0.000000] ACPI: Early table checksum verification disabled [ 0.000000] ACPI: RSDP 0x00000000000F5950 000014 (v00 BOCHS ) [ 0.000000] ACPI: RSDT 0x00000000BFFE1BB7 000034 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACP 0x00000000BFFE1A53 000074 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: DSDT 0x00000000BFFE0040 001A13 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACS 0x00000000BFFE0000 000040 [ 0.000000] ACPI: APIC 0x00000000BFFE1AC7 000090 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: HPET 0x00000000BFFE1B57 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: WAET 0x00000000BFFE1B8F 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: Reserving FACP table memory at [mem 0xbffe1a53-0xbffe1ac6] [ 0.000000] ACPI: Reserving DSDT table memory at [mem 0xbffe0040-0xbffe1a52] [ 0.000000] ACPI: Reserving FACS table memory at [mem 0xbffe0000-0xbffe003f] [ 0.000000] ACPI: Reserving APIC table memory at [mem 0xbffe1ac7-0xbffe1b56] [ 0.000000] ACPI: Reserving HPET table memory at [mem 0xbffe1b57-0xbffe1b8e] [ 0.000000] ACPI: Reserving WAET table memory at [mem 0xbffe1b8f-0xbffe1bb6] [ 0.000000] No NUMA configuration found [ 0.000000] Faking a node at [mem 0x0000000000000000-0x0000000146dfffff] [ 0.000000] NODE_DATA(0) allocated [mem 0x1465a3000-0x1465cdfff] [ 0.000000] Reserving 256MB of memory at 2752MB for crashkernel (System RAM: 4205MB) [ 0.000000] Zone ranges: [ 0.000000] DMA [mem 0x0000000000001000-0x0000000000ffffff] [ 0.000000] DMA32 [mem 0x0000000001000000-0x00000000ffffffff] [ 0.000000] Normal [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Device empty [ 0.000000] Movable zone start for each node [ 0.000000] Early memory node ranges [ 0.000000] node 0: [mem 0x0000000000001000-0x000000000009efff] [ 0.000000] node 0: [mem 0x0000000000100000-0x00000000bffcdfff] [ 0.000000] node 0: [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Zeroed struct page in unavailable ranges: 4756 pages [ 0.000000] Initmem setup node 0 [mem 0x0000000000001000-0x0000000146dfffff] [ 0.000000] ACPI: PM-Timer IO Port: 0x608 [ 0.000000] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1]) [ 0.000000] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23 [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level) [ 0.000000] Using ACPI (MADT) for SMP configuration information [ 0.000000] ACPI: HPET id: 0x8086a201 base: 0xfed00000 [ 0.000000] TSC deadline timer available [ 0.000000] smpboot: Allowing 4 CPUs, 0 hotplug CPUs [ 0.000000] kvm-guest: KVM setup pv remote TLB flush [ 0.000000] kvm-guest: setup PV sched yield [ 0.000000] PM: Registered nosave memory: [mem 0x00000000-0x00000fff] [ 0.000000] PM: Registered nosave memory: [mem 0x0009f000-0x0009ffff] [ 0.000000] PM: Registered nosave memory: [mem 0x000a0000-0x000effff] [ 0.000000] PM: Registered nosave memory: [mem 0x000f0000-0x000fffff] [ 0.000000] PM: Registered nosave memory: [mem 0xbffce000-0xbfffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xc0000000-0xfeffbfff] [ 0.000000] PM: Registered nosave memory: [mem 0xfeffc000-0xfeffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xff000000-0xfffbffff] [ 0.000000] PM: Registered nosave memory: [mem 0xfffc0000-0xffffffff] [ 0.000000] [mem 0xc0000000-0xfeffbfff] available for PCI devices [ 0.000000] Booting paravirtualized kernel on KVM [ 0.000000] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns [ 0.000000] setup_percpu: NR_CPUS:8192 nr_cpumask_bits:4 nr_cpu_ids:4 nr_node_ids:1 [ 0.000000] percpu: Embedded 63 pages/cpu s221184 r8192 d28672 u524288 [ 0.000000] kvm-guest: PV spinlocks enabled [ 0.000000] PV qspinlock hash table entries: 256 (order: 0, 4096 bytes, linear) [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 1059606 [ 0.000000] Policy zone: Normal [ 0.000000] Kernel command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] Specific versions of hardware are certified with Red Hat Enterprise Linux 8. Please see the list of hardware certified with Red Hat Enterprise Linux 8 at https://catalog.redhat.com. [ 0.000000] audit: disabled (until reboot) [ 0.000000] software IO TLB: area num 4. [ 0.000000] Memory: 2829652K/4306352K available (18435K kernel code, 11221K rwdata, 7248K rodata, 2908K init, 18040K bss, 524580K reserved, 0K cma-reserved) [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1 [ 0.000000] kmemleak: Kernel memory leak detector disabled [ 0.000000] ftrace: allocating 41240 entries in 162 pages [ 0.000000] ftrace: allocated 162 pages with 3 groups [ 0.000000] rcu: Hierarchical RCU implementation. [ 0.000000] rcu: RCU event tracing is enabled. [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=8192 to nr_cpu_ids=4. [ 0.000000] rcu: RCU callback double-/use-after-free debug enabled. [ 0.000000] Rude variant of Tasks RCU enabled. [ 0.000000] Tracing variant of Tasks RCU enabled. [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies. [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4 [ 0.000000] NR_IRQS: 524544, nr_irqs: 456, preallocated irqs: 16 [ 0.000000] random: get_random_bytes called from start_kernel+0x622/0x9a8 with crng_init=0 [ 0.001000] Console: colour *CGA 80x25 [ 0.001000] printk: console [ttyS1] enabled [ 0.001000] ACPI: Core revision 20220331 [ 0.001000] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns [ 0.001010] APIC: Switch to symmetric I/O mode setup [ 0.002000] x2apic enabled [ 0.002008] Switched APIC routing to physical x2apic. [ 0.003009] kvm-guest: setup PV IPIs [ 0.006462] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1 [ 0.007000] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 0.007022] Calibrating delay loop (skipped) preset value.. 4799.99 BogoMIPS (lpj=2399998) [ 0.008010] pid_max: default: 32768 minimum: 301 [ 0.009129] LSM: Security Framework initializing [ 0.010039] Yama: becoming mindful. [ 0.010859] SELinux: Initializing. [ 0.011057] *** VALIDATE selinux *** [ 0.019768] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, vmalloc) [ 0.024500] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, vmalloc) [ 0.025166] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.027088] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.028150] *** VALIDATE tmpfs *** [ 0.029477] *** VALIDATE proc *** [ 0.030000] *** VALIDATE cgroup *** [ 0.030011] *** VALIDATE cgroup2 *** [ 0.032162] x86/cpu: User Mode Instruction Prevention (UMIP) activated [ 0.033196] Last level iTLB entries: 4KB 0, 2MB 0, 4MB 0 [ 0.034008] Last level dTLB entries: 4KB 0, 2MB 0, 4MB 0, 1GB 0 [ 0.035045] Spectre V2 : User space: Vulnerable [ 0.036009] Speculative Store Bypass: Vulnerable [ 0.039676] debug: unmapping init [mem 0xffffffffbd659000-0xffffffffbd660fff] [ 0.041000] smpboot: CPU0: Intel(R) Xeon(R) CPU E5-2695 v2 @ 2.40GHz (family: 0x6, model: 0x3e, stepping: 0x4) [ 0.041749] Performance Events: IvyBridge events, full-width counters, Intel PMU driver. [ 0.042020] ... version: 2 [ 0.043009] ... bit width: 48 [ 0.044006] ... generic registers: 4 [ 0.045007] ... value mask: 0000ffffffffffff [ 0.046011] ... max period: 00007fffffffffff [ 0.047010] ... fixed-purpose events: 3 [ 0.048011] ... event mask: 000000070000000f [ 0.049274] rcu: Hierarchical SRCU implementation. [ 0.051482] smp: Bringing up secondary CPUs ... [ 0.053463] x86: Booting SMP configuration: [ 0.054021] .... node #0, CPUs: #1 #2 #3 [ 0.060530] smp: Brought up 1 node, 4 CPUs [ 0.062009] smpboot: Max logical packages: 1 [ 0.063013] smpboot: Total of 4 processors activated (19199.98 BogoMIPS) [ 0.093887] node 0 deferred pages initialised in 28ms [ 0.101166] devtmpfs: initialized [ 0.102218] x86/mm: Memory block size: 128MB [ 0.105157] gcov: version magic: 0x41383552 [ 0.107573] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns [ 0.108105] futex hash table entries: 1024 (order: 4, 65536 bytes, vmalloc) [ 0.109233] pinctrl core: initialized pinctrl subsystem [ 0.110228] [ 0.110776] ************************************************************* [ 0.111015] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.112015] ** ** [ 0.113012] ** IOMMU DebugFS SUPPORT HAS BEEN ENABLED IN THIS KERNEL ** [ 0.114013] ** ** [ 0.115010] ** This means that this kernel is built to expose internal ** [ 0.116012] ** IOMMU data structures, which may compromise security on ** [ 0.117011] ** your system. ** [ 0.118012] ** ** [ 0.119013] ** If you see this message and you are not debugging the ** [ 0.120013] ** kernel, report this immediately to your vendor! ** [ 0.121013] ** ** [ 0.122011] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.123011] ************************************************************* [ 0.125651] NET: Registered protocol family 16 [ 0.126456] DMA: preallocated 512 KiB GFP_KERNEL pool for atomic allocations [ 0.127513] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations [ 0.128063] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations [ 0.130079] cpuidle: using governor menu [ 0.131937] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5 [ 0.134504] PCI: Using configuration type 1 for base access [ 0.136126] core: PMU erratum BJ122, BV98, HSD29 worked around, HT is on [ 0.148324] HugeTLB registered 1.00 GiB page size, pre-allocated 0 pages [ 0.149018] HugeTLB registered 2.00 MiB page size, pre-allocated 0 pages [ 0.158087] cryptd: max_cpu_qlen set to 1000 [ 0.163576] ACPI: Added _OSI(Module Device) [ 0.167032] ACPI: Added _OSI(Processor Device) [ 0.169031] ACPI: Added _OSI(3.0 _SCP Extensions) [ 0.171023] ACPI: Added _OSI(Processor Aggregator Device) [ 0.177000] ACPI: 1 ACPI AML tables successfully acquired and loaded [ 0.185571] ACPI: Interpreter enabled [ 0.187078] ACPI: PM: (supports S0 S3 S4 S5) [ 0.190454] ACPI: Using IOAPIC for interrupt routing [ 0.192862] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug [ 0.197448] ACPI: Enabled 2 GPEs in block 00 to 0F [ 0.208161] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff]) [ 0.210033] acpi PNP0A03:00: _OSC: OS supports [ASPM ClockPM Segments MSI HPX-Type3] [ 0.212013] acpi PNP0A03:00: _OSC: not requesting OS control; OS requires [ExtendedConfig ASPM ClockPM MSI] [ 0.216121] acpi PNP0A03:00: fail to add MMCONFIG information, can't access extended PCI configuration space under this bridge. [ 0.220353] acpiphp: Slot [2] registered [ 0.221088] acpiphp: Slot [3] registered [ 0.223074] acpiphp: Slot [4] registered [ 0.224108] acpiphp: Slot [5] registered [ 0.226120] acpiphp: Slot [6] registered [ 0.227086] acpiphp: Slot [7] registered [ 0.228087] acpiphp: Slot [8] registered [ 0.229102] acpiphp: Slot [9] registered [ 0.231112] acpiphp: Slot [10] registered [ 0.233182] acpiphp: Slot [11] registered [ 0.235118] acpiphp: Slot [12] registered [ 0.237131] acpiphp: Slot [13] registered [ 0.239186] acpiphp: Slot [14] registered [ 0.241096] acpiphp: Slot [15] registered [ 0.243086] acpiphp: Slot [16] registered [ 0.244100] acpiphp: Slot [17] registered [ 0.245082] acpiphp: Slot [18] registered [ 0.247104] acpiphp: Slot [19] registered [ 0.248084] acpiphp: Slot [20] registered [ 0.250088] acpiphp: Slot [21] registered [ 0.251178] acpiphp: Slot [22] registered [ 0.253086] acpiphp: Slot [23] registered [ 0.254205] acpiphp: Slot [24] registered [ 0.256092] acpiphp: Slot [25] registered [ 0.257135] acpiphp: Slot [26] registered [ 0.259139] acpiphp: Slot [27] registered [ 0.261121] acpiphp: Slot [28] registered [ 0.262000] acpiphp: Slot [29] registered [ 0.262018] acpiphp: Slot [30] registered [ 0.263000] acpiphp: Slot [31] registered [ 0.264049] PCI host bridge to bus 0000:00 [ 0.265014] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window] [ 0.268019] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window] [ 0.270019] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window] [ 0.273134] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window] [ 0.276030] pci_bus 0000:00: root bus resource [mem 0x180000000-0x1ffffffff window] [ 0.280025] pci_bus 0000:00: root bus resource [bus 00-ff] [ 0.282155] pci 0000:00:00.0: [8086:1237] type 00 class 0x060000 [ 0.287200] pci 0000:00:01.0: [8086:7000] type 00 class 0x060100 [ 0.292022] pci 0000:00:01.1: [8086:7010] type 00 class 0x010180 [ 0.302710] pci 0000:00:01.1: reg 0x20: [io 0xc320-0xc32f] [ 0.308025] pci 0000:00:01.1: legacy IDE quirk: reg 0x10: [io 0x01f0-0x01f7] [ 0.310012] pci 0000:00:01.1: legacy IDE quirk: reg 0x14: [io 0x03f6] [ 0.312014] pci 0000:00:01.1: legacy IDE quirk: reg 0x18: [io 0x0170-0x0177] [ 0.315015] pci 0000:00:01.1: legacy IDE quirk: reg 0x1c: [io 0x0376] [ 0.318442] pci 0000:00:01.3: [8086:7113] type 00 class 0x068000 [ 0.319000] pci 0000:00:01.3: quirk: [io 0x0600-0x063f] claimed by PIIX4 ACPI [ 0.321030] pci 0000:00:01.3: quirk: [io 0x0700-0x070f] claimed by PIIX4 SMB [ 0.323546] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 [ 0.329013] pci 0000:00:02.0: reg 0x10: [io 0xc300-0xc31f] [ 0.342014] pci 0000:00:02.0: reg 0x20: [mem 0xfebe4000-0xfebe7fff 64bit pref] [ 0.348015] pci 0000:00:02.0: reg 0x30: [mem 0xfeb80000-0xfebbffff pref] [ 0.353756] pci 0000:00:05.0: [1af4:1001] type 00 class 0x010000 [ 0.361016] pci 0000:00:05.0: reg 0x10: [io 0xc000-0xc07f] [ 0.367020] pci 0000:00:05.0: reg 0x14: [mem 0xfebc0000-0xfebc0fff] [ 0.386014] pci 0000:00:05.0: reg 0x20: [mem 0xfebe8000-0xfebebfff 64bit pref] [ 0.398000] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 [ 0.405014] pci 0000:00:06.0: reg 0x10: [io 0xc080-0xc0ff] [ 0.413015] pci 0000:00:06.0: reg 0x14: [mem 0xfebc1000-0xfebc1fff] [ 0.433016] pci 0000:00:06.0: reg 0x20: [mem 0xfebec000-0xfebeffff 64bit pref] [ 0.455000] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 [ 0.463017] pci 0000:00:07.0: reg 0x10: [io 0xc100-0xc17f] [ 0.470018] pci 0000:00:07.0: reg 0x14: [mem 0xfebc2000-0xfebc2fff] [ 0.502026] pci 0000:00:07.0: reg 0x20: [mem 0xfebf0000-0xfebf3fff 64bit pref] [ 0.525720] pci 0000:00:08.0: [1af4:1001] type 00 class 0x010000 [ 0.530000] pci 0000:00:08.0: reg 0x10: [io 0xc180-0xc1ff] [ 0.534017] pci 0000:00:08.0: reg 0x14: [mem 0xfebc3000-0xfebc3fff] [ 0.553028] pci 0000:00:08.0: reg 0x20: [mem 0xfebf4000-0xfebf7fff 64bit pref] [ 0.566000] pci 0000:00:09.0: [1af4:1001] type 00 class 0x010000 [ 0.573017] pci 0000:00:09.0: reg 0x10: [io 0xc200-0xc27f] [ 0.580016] pci 0000:00:09.0: reg 0x14: [mem 0xfebc4000-0xfebc4fff] [ 0.605014] pci 0000:00:09.0: reg 0x20: [mem 0xfebf8000-0xfebfbfff 64bit pref] [ 0.624465] pci 0000:00:0a.0: [1af4:1001] type 00 class 0x010000 [ 0.633014] pci 0000:00:0a.0: reg 0x10: [io 0xc280-0xc2ff] [ 0.641014] pci 0000:00:0a.0: reg 0x14: [mem 0xfebc5000-0xfebc5fff] [ 0.671017] pci 0000:00:0a.0: reg 0x20: [mem 0xfebfc000-0xfebfffff 64bit pref] [ 0.695341] ACPI: PCI: Interrupt link LNKA configured for IRQ 10 [ 0.698445] ACPI: PCI: Interrupt link LNKB configured for IRQ 10 [ 0.700533] ACPI: PCI: Interrupt link LNKC configured for IRQ 11 [ 0.706391] ACPI: PCI: Interrupt link LNKD configured for IRQ 11 [ 0.708202] ACPI: PCI: Interrupt link LNKS configured for IRQ 9 [ 0.715029] iommu: Default domain type: Passthrough [ 0.716751] SCSI subsystem initialized [ 0.718179] ACPI: bus type USB registered [ 0.720108] usbcore: registered new interface driver usbfs [ 0.721861] usbcore: registered new interface driver hub [ 0.724077] usbcore: registered new device driver usb [ 0.727178] pps_core: LinuxPPS API ver. 1 registered [ 0.730011] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti [ 0.737057] PTP clock support registered [ 0.740120] EDAC MC: Ver: 3.0.0 [ 0.743030] PCI: Using ACPI for IRQ routing [ 0.744871] NetLabel: Initializing [ 0.746013] NetLabel: domain hash size = 128 [ 0.748006] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO [ 0.749223] NetLabel: unlabeled traffic allowed by default [ 0.752039] vgaarb: loaded [ 0.753254] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0 [ 0.755012] hpet0: 3 comparators, 64-bit 100.000000 MHz counter [ 0.763970] clocksource: Switched to clocksource kvm-clock [ 0.921171] VFS: Disk quotas dquot_6.6.0 [ 0.922477] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) [ 0.924653] *** VALIDATE ramfs *** [ 0.925755] *** VALIDATE hugetlbfs *** [ 0.927116] pnp: PnP ACPI init [ 0.929823] pnp: PnP ACPI: found 6 devices [ 0.947251] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns [ 0.950094] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window] [ 0.952031] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window] [ 0.953877] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window] [ 0.956040] pci_bus 0000:00: resource 7 [mem 0xc0000000-0xfebfffff window] [ 0.958257] pci_bus 0000:00: resource 8 [mem 0x180000000-0x1ffffffff window] [ 0.960841] NET: Registered protocol family 2 [ 0.962932] IP idents hash table entries: 131072 (order: 8, 1048576 bytes, vmalloc) [ 0.967260] tcp_listen_portaddr_hash hash table entries: 4096 (order: 5, 163840 bytes, vmalloc) [ 0.970779] TCP established hash table entries: 65536 (order: 7, 524288 bytes, vmalloc) [ 0.975195] TCP bind hash table entries: 65536 (order: 9, 2097152 bytes, vmalloc) [ 0.978824] TCP: Hash tables configured (established 65536 bind 65536) [ 0.982839] MPTCP token hash table entries: 8192 (order: 6, 393216 bytes, vmalloc) [ 0.991839] UDP hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 1.001444] UDP-Lite hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 1.007829] NET: Registered protocol family 1 [ 1.018216] RPC: Registered named UNIX socket transport module. [ 1.020229] RPC: Registered udp transport module. [ 1.027645] RPC: Registered tcp transport module. [ 1.036616] RPC: Registered tcp NFSv4.1 backchannel transport module. [ 1.058689] NET: Registered protocol family 44 [ 1.060321] pci 0000:00:00.0: Limiting direct PCI/PCI transfers [ 1.078387] pci 0000:00:00.0: quirk_natoma+0x0/0x20 took 17643 usecs [ 1.092957] pci 0000:00:01.0: PIIX3: Enabling Passive Release [ 1.100858] pci 0000:00:01.0: Activating ISA DMA hang workarounds [ 1.109865] PCI: CLS 0 bytes, default 64 [ 1.115258] Unpacking initramfs... [ 3.148823] debug: unmapping init [mem 0xffff8c44bcc54000-0xffff8c44bffbffff] [ 3.155586] PCI-DMA: Using software bounce buffering for IO (SWIOTLB) [ 3.158329] software IO TLB: mapped [mem 0x00000000a8000000-0x00000000ac000000] (64MB) [ 3.162207] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 4.027530] Initialise system trusted keyrings [ 4.029973] Key type blacklist registered [ 4.032654] workingset: timestamp_bits=36 max_order=20 bucket_order=0 [ 4.046075] zbud: loaded [ 4.049554] *** VALIDATE nfs *** [ 4.052277] *** VALIDATE nfs4 *** [ 4.055233] pstore: using deflate compression [ 4.061210] Platform Keyring initialized [ 4.257802] NET: Registered protocol family 38 [ 4.259947] Key type asymmetric registered [ 4.263143] Asymmetric key parser 'x509' registered [ 4.267514] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 247) [ 4.273901] io scheduler mq-deadline registered [ 4.277691] io scheduler kyber registered [ 4.280386] io scheduler bfq registered [ 4.284106] atomic64_test: passed for x86-64 platform with CX8 and with SSE [ 4.287337] shpchp: Standard Hot Plug PCI Controller Driver version: 0.4 [ 4.303855] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0 [ 4.310374] ACPI: Power Button [PWRF] [ 4.486611] ACPI: \_SB_.LNKB: Enabled at IRQ 10 [ 4.646207] ACPI: \_SB_.LNKA: Enabled at IRQ 11 [ 4.900958] ACPI: \_SB_.LNKC: Enabled at IRQ 11 [ 5.028081] ACPI: \_SB_.LNKD: Enabled at IRQ 10 [ 5.297301] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled [ 5.332614] 00:03: ttyS1 at I/O 0x2f8 (irq = 3, base_baud = 115200) is a 16550A [ 5.375277] 00:04: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A [ 5.386532] Non-volatile memory driver v1.3 [ 5.389240] Linux agpgart interface v0.103 [ 5.443957] virtio_blk virtio1: [vda] 133600 512-byte logical blocks (68.4 MB/65.2 MiB) [ 5.447739] vda: detected capacity change from 0 to 68403200 [ 5.463419] virtio_blk virtio2: [vdb] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) [ 5.467766] vdb: detected capacity change from 0 to 1073741824 [ 5.485359] virtio_blk virtio3: [vdc] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 5.488135] vdc: detected capacity change from 0 to 2621440000 [ 5.504769] virtio_blk virtio4: [vdd] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 5.507927] vdd: detected capacity change from 0 to 2621440000 [ 5.543891] virtio_blk virtio5: [vde] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 5.550975] vde: detected capacity change from 0 to 4294967296 [ 5.595201] virtio_blk virtio6: [vdf] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 5.605860] vdf: detected capacity change from 0 to 4294967296 [ 5.621655] libphy: Fixed MDIO Bus: probed [ 5.632205] usbcore: registered new interface driver usbserial_generic [ 5.638555] usbserial: USB Serial support registered for generic [ 5.644947] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12 [ 5.657572] serio: i8042 KBD port at 0x60,0x64 irq 1 [ 5.662125] serio: i8042 AUX port at 0x60,0x64 irq 12 [ 5.668532] mousedev: PS/2 mouse device common for all mice [ 5.681461] rtc_cmos 00:05: RTC can wake from S4 [ 5.684579] rtc_cmos 00:05: registered as rtc0 [ 5.686304] rtc_cmos 00:05: alarms up to one day, y3k, 242 bytes nvram, hpet irqs [ 5.693087] intel_pstate: CPU model not supported [ 5.694968] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input1 [ 5.701692] hid: raw HID events driver (C) Jiri Kosina [ 5.710339] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input4 [ 5.717127] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input3 [ 5.731851] usbcore: registered new interface driver usbhid [ 5.733705] usbhid: USB HID core driver [ 5.735632] drop_monitor: Initializing network drop monitor service [ 5.738714] Initializing XFRM netlink socket [ 5.742923] NET: Registered protocol family 10 [ 5.747878] Segment Routing with IPv6 [ 5.751781] NET: Registered protocol family 17 [ 5.757340] mpls_gso: MPLS GSO support [ 5.765127] RAS: Correctable Errors collector initialized. [ 5.772342] AVX version of gcm_enc/dec engaged. [ 5.773953] AES CTR mode by8 optimization enabled [ 6.013288] sched_clock: Marking stable (6013220324, 0)->(7229361316, -1216140992) [ 6.018513] registered taskstats version 1 [ 6.027199] Loading compiled-in X.509 certificates [ 6.029476] zswap: loaded using pool lzo/zbud [ 6.074237] Key type big_key registered [ 6.102206] Key type encrypted registered [ 6.103979] ima: No TPM chip found, activating TPM-bypass! [ 6.107322] ima: Allocated hash algorithm: sha1 [ 6.111309] ima: No architecture policies found [ 6.113450] evm: Initialising EVM extended attributes: [ 6.115801] evm: security.selinux [ 6.117801] evm: security.ima [ 6.119225] evm: security.capability [ 6.121398] evm: HMAC attrs: 0x1 [ 6.125789] rtc_cmos 00:05: setting system clock to 2025-11-03 10:39:47 UTC (1762166387) [ 6.141850] debug: unmapping init [mem 0xffffffffbe603000-0xffffffffbe7fffff] [ 6.144882] debug: unmapping init [mem 0xffffffffbd382000-0xffffffffbd658fff] [ 6.154047] Write protecting the kernel read-only data: 28672k [ 6.157617] debug: unmapping init [mem 0xffffffffbba03000-0xffffffffbbbfffff] [ 6.161337] debug: unmapping init [mem 0xffffffffbc314000-0xffffffffbc3fffff] [ 6.203408] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 6.211650] systemd[1]: Detected virtualization kvm. [ 6.214379] systemd[1]: Detected architecture x86-64. [ 6.216838] systemd[1]: Running in initial RAM disk. Welcome to Rocky Linux 8.10 (Green Obsidian) dracut-049-233.git20240115.el8 (Initramfs)! [ 6.247292] systemd[1]: No hostname configured. [ 6.249020] systemd[1]: Set hostname to . [ 6.250534] random: systemd: uninitialized urandom read (16 bytes read) [ 6.252826] systemd[1]: Initializing machine ID from random generator. [ 6.540703] random: systemd: uninitialized urandom read (16 bytes read) [ 6.546584] systemd[1]: Reached target Slices. [ OK ] Reached target Slices. [ 6.553784] random: systemd: uninitialized urandom read (16 bytes read) [ 6.555885] systemd[1]: Reached target Swap. [ OK ] Reached target Swap. [ 6.564808] systemd[1]: Reached target Local File Systems. [ OK ] Reached target Local File Systems. [ OK ] Reached target Timers. [ OK ] Listening on Journal Socket. [ OK ] Started Memstrack Anylazing Service. Starting Setup Virtual Console... Starting Apply Kernel Variables... [ OK ] Listening on Journal Socket (/dev/log). [ OK ] Listening on udev Kernel Socket. Starting Create list of required st…ce nodes for the current kernel... [ OK ] Reached target Initrd Root Device. [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Reached target Paths. Starting Journal Service... [ OK ] Reached target Local Encrypted Volumes. Starting Create Volatile Files and Directories... [ OK ] Listening on udev Control Socket. [ OK ] Reached target Sockets. [ OK ] Started Setup Virtual Console. [ OK ] Started Apply Kernel Variables. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Started Create Volatile Files and Directories. Starting Create Static Device Nodes in /dev... Starting dracut cmdline hook... [ OK ] Started Create Static Device Nodes in /dev. [ OK ] Started Journal Service. [ OK ] Started dracut cmdline hook. Starting dracut pre-udev hook... [ 8.463978] device-mapper: uevent: version 1.0.3 [ 8.465981] device-mapper: ioctl: 4.46.0-ioctl (2022-02-22) initialised: dm-devel@redhat.com [ OK ] Started dracut pre-udev hook. Starting udev Kernel Device Manager... [ OK ] Started udev Kernel Device Manager. Starting dracut pre-trigger hook... [ OK ] Started dracut pre-trigger hook. Starting udev Coldplug all Devices... Mounting Kernel Configuration File System... [ OK ] Mounted Kernel Configuration File System. [ OK ] Started udev Coldplug all Devices. Starting dracut initqueue hook... [ OK ] Reached target System Initialization. [ OK ] Reached target Basic System. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. [ 10.136211] virtio_net virtio0 ens2: renamed from eth0 [ 10.302165] random: fast init done [ 10.832191] scsi host0: ata_piix [ 11.113018] scsi host1: ata_piix [ 11.114424] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc320 irq 14 [ 11.116818] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc328 irq 15 [ 15.645885] random: crng init done [ 15.646994] random: 7 urandom warning(s) missed due to ratelimiting [ 18.210016] dracut-initqueue[589]: RTNETLINK answers: File exists Starting nbd nbd0... [ OK ] Started nbd nbd0. [ OK ] Started dracut initqueue hook. Mounting /sysroot... [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. [ 20.219472] EXT4-fs (nbd0): mounted filesystem with ordered data mode. Opts: (null) [ OK ] Mounted /sysroot. [ OK ] Reached target Initrd Root File System. Starting Reload Configuration from the Real Root... [ OK ] Started Reload Configuration from the Real Root. [ OK ] Reached target Initrd File Systems. [ OK ] Reached target Initrd Default Target. Starting dracut pre-pivot and cleanup hook... [ OK ] Started dracut pre-pivot and cleanup hook. Starting Cleaning Up and Shutting Down Daemons... [ OK ] Stopped dracut pre-pivot and cleanup hook. [ OK ] Stopped target Initrd Default Target. [ OK ] Stopped target Timers. [ OK ] Stopped target Remote File Systems. Stopping Hardware RNG Entropy Gatherer Daemon... [ OK ] Stopped target Remote File Systems (Pre). [ OK ] Stopped dracut initqueue hook. [ OK ] Stopped target Initrd Root Device. [ OK ] Stopped Hardware RNG Entropy Gatherer Daemon. [ OK ] Stopped target Basic System. [ OK ] Stopped target Paths. [ OK ] Stopped target Slices. [ OK ] Stopped target System Initialization. [ OK ] Stopped Create Volatile Files and Directories. [ OK ] Stopped Apply Kernel Variables. [ OK ] Stopped udev Coldplug all Devices. [ OK ] Stopped dracut pre-trigger hook. Stopping udev Kernel Device Manager... [ OK ] Stopped target Local File Systems. [ OK ] Stopped target Swap. [ OK ] Stopped target Local Encrypted Volumes. [ OK ] Stopped Dispatch Password Requests to Console Directory Watch. [ OK ] Stopped target Sockets. [ OK ] Stopped udev Kernel Device Manager. [ OK ] Started Cleaning Up and Shutting Down Daemons. [ OK ] Stopped Create Static Device Nodes in /dev. [ OK ] Stopped Create list of required sta…vice nodes for the current kernel. [ OK ] Stopped dracut pre-udev hook. [ OK ] Stopped dracut cmdline hook. [ OK ] Closed udev Control Socket. [ OK ] Closed udev Kernel Socket. Starting Cleanup udevd DB... [ OK ] Started Cleanup udevd DB. [ OK ] Reached target Switch Root. Starting Switch Root... [ 22.281412] printk: systemd: 26 output lines suppressed due to ratelimiting [ 22.910935] SELinux: Disabled at runtime. [ 23.049707] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 23.070042] systemd[1]: Detected virtualization kvm. [ 23.071671] systemd[1]: Detected architecture x86-64. Welcome to Rocky Linux 8.10 (Green Obsidian)! [ 24.687847] systemd[1]: initrd-switch-root.service: Succeeded. [ 24.696732] systemd[1]: Stopped Switch Root. [ OK ] Stopped Switch Root. [ 24.711254] systemd[1]: systemd-journald.service: Service has no hold-off time (RestartSec=0), scheduling restart. [ 24.715259] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. [ 24.718493] systemd[1]: Stopped Journal Service. [ OK ] Stopped Journal Service. [ 24.747946] systemd[1]: Starting Journal Service... Starting Journal Service... [ 24.765016] systemd[1]: Created slice system-sshd\x2dkeygen.slice. [ OK ] Created slice system-sshd\x2dkeygen.slice. Starting Remount Root and Kernel File Systems... [ OK ] Listening on udev Control Socket. [ OK ] Listening on initctl Compatibility Named Pipe. [ OK ] Stopped target Switch Root. Activating swap /dev/disk/by-label/SWAP... [ OK ] Created slice system-serial\x2dgetty.slice. [ OK ] Reached target rpc_pipefs.target. Mounting POSIX Message Queue File System... Starting Apply Kernel Variables... [ OK ] Started Forward Password Requests to Wall Directory Watch. Mounting Huge Pages File System... [ OK ] Listening on udev Kernel Socket. Starting udev Coldplug all Devices... [ 24.996958] Adding 1048572k swap on /dev/vdb. Priority:-2 extents:1 across:1048572k FS Starting Create list of required st…ce nodes for the current kernel... [ OK ] Listening on Process Core Dump Socket. [ OK ] Listening on RPCbind Server Activation Socket. [ OK ] Reached target RPC Port Mapper. Mounting Kernel Debug File System... [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Reached target Local Encrypted Volumes. [ OK ] Reached target Paths. [ OK ] Stopped target Initrd File Systems. [FAILED] Failed to set up automount Arbitrar…rmats File System Automount Point. See 'systemctl status proc-sys-fs-binfmt_misc.automount' for details. [ OK ] Created slice system-getty.slice. [ OK ] Created slice User and Session Slice. [ OK ] Reached target Slices. [ OK ] Stopped target Initrd Root File System. [ OK ] Started Journal Service. [FAILED] Failed to start Remount Root and Kernel File Systems. See 'systemctl status systemd-remount-fs.service' for details. [ OK ] Activated swap /dev/disk/by-label/SWAP. [ OK ] Mounted POSIX Message Queue File System. [ OK ] Started Apply Kernel Variables. [ OK ] Mounted Huge Pages File System. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Mounted Kernel Debug File System. [ OK ] Reached target Swap. Starting Configure read-only root support... Starting Create Static Device Nodes in /dev... Starting Flush Journal to Persistent Storage... [ OK ] Started Flush Journal to Persistent Storage. [ OK ] Started udev Coldplug all Devices. [ OK ] Started Create Static Device Nodes in /dev. Starting udev Kernel Device Manager... [ OK ] Reached target Local File Systems (Pre). Mounting /mnt... Mounting /home/green/git/lustre-release... [ OK ] Mounted /mnt. [ 25.979820] squashfs: version 4.0 (2009/01/31) Phillip Lougher [ OK ] Mounted /home/green/git/lustre-release. [ 26.214950] hrtimer: interrupt took 4283320 ns [ OK ] Started udev Kernel Device Manager. [ 26.733785] input: PC Speaker as /devices/platform/pcspkr/input/input5 [ 26.794168] piix4_smbus 0000:00:01.3: SMBus Host Controller at 0x700, revision 0 [ 28.036230] RAPL PMU: API unit is 2^-32 Joules, 0 fixed counters, 10737418240 ms ovfl timer [ 28.413193] EDAC sbridge: Ver: 1.1.2 [* ] A start job is running for Configur…-only root support (7s / no limit) [** ] A start job is running for Configur…-only root support (7s / no limit) [*** ] A start job is running for Configur…-only root support (8s / no limit) [ *** ] A start job is running for Configur…-only root support (8s / no limit)[ 33.496470] Key type dns_resolver registered [ *** ] A start job is running for Configur…-only root support (9s / no limit)[ 34.365200] NFS: Registering the id_resolver key type [ 34.367683] Key type id_resolver registered [ 34.369984] Key type id_legacy registered [ ***] A start job is running for Configur…-only root support (9s / no limit) [ **] A start job is running for Configur…only root support (10s / no limit) [ OK ] Started Configure read-only root support. Starting Load/Save Random Seed... [ OK ] Reached target Local File Systems. Starting Rebuild Dynamic Linker Cache... Starting Mark the need to relabel after reboot... Starting Create Volatile Files and Directories... [ OK ] Started Load/Save Random Seed. [ OK ] Started Mark the need to relabel after reboot. [ OK ] Started Create Volatile Files and Directories. Starting Update UTMP about System Boot/Shutdown... Starting RPC Bind... [ OK ] Started Update UTMP about System Boot/Shutdown. [ OK ] Started RPC Bind. [ OK ] Started Rebuild Dynamic Linker Cache. Starting Update is Completed... [ OK ] Started Update is Completed. [ OK ] Reached target System Initialization. [ OK ] Listening on D-Bus System Message Bus Socket. [ OK ] Reached target Sockets. [ OK ] Started daily update of the root trust anchor for DNSSEC. [ OK ] Started dnf makecache --timer. [ OK ] Started Daily Cleanup of Temporary Directories. [ OK ] Reached target Timers. [ OK ] Reached target Basic System. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. Starting Restore /run/initramfs on shutdown... [ OK ] Started D-Bus System Message Bus. Starting Network Manager... Starting Login Service... [ OK ] Started irqbalance daemon. [ OK ] Reached target sshd-keygen.target. [ OK ] Started Restore /run/initramfs on shutdown. [ OK ] Started Network Manager. [ OK ] Reached target Network. Starting Dynamic System Tuning Daemon... Starting OpenSSH server daemon... Starting GSSAPI Proxy Daemon... Starting Network Manager Wait Online... [ OK ] Started Login Service. Starting Hostname Service... [ OK ] Started GSSAPI Proxy Daemon. [ OK ] Reached target NFS client services. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Starting Permit User Sessions... [ OK ] Started OpenSSH server daemon. [ OK ] Started Permit User Sessions. [ OK ] Started Serial Getty on ttyS1. [ OK ] Started Getty on tty1. [ OK ] Started Serial Getty on ttyS0. [ OK ] Reached target Login Prompts. [ OK ] Started Command Scheduler. [ OK ] Started Hostname Service. Starting Network Manager Script Dispatcher Service... [ OK ] Started Network Manager Script Dispatcher Service. [ OK ] Started Network Manager Wait Online. [ OK ] Reached target Network is Online. Starting Crash recovery kernel arming... Starting System Logging Service... Starting Notify NFS peers of a restart... [ OK ] Started Notify NFS peers of a restart. [ OK ] Started System Logging Service. Starting Authorization Manager... Rocky Linux 8.10 (Green Obsidian) Kernel 4.18.0rh8.10-debug on an x86_64 oleg247-server login: [ 71.977672] libcfs: loading out-of-tree module taints kernel. [ 72.000246] Key type ._llcrypt registered [ 72.001863] Key type .llcrypt registered [ 72.061343] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_hostid [ 81.138561] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing load_modules_local [ 81.860131] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 81.869470] alg: No test for adler32 (adler32-zlib) [ 82.990365] Lustre: Lustre: Build Version: 2.16.59_50_g4204c43 [ 83.374134] LNet: Added LNI 192.168.202.147@tcp [8/256/0/180] [ 84.999799] Key type lgssc registered [ 85.766267] Lustre: Echo OBD driver; http://www.lustre.org/ [ 95.490400] ZFS: Loaded module v2.3.2-1, ZFS pool version 5000, ZFS filesystem version 5 [ 96.020767] LDISKFS-fs (vdc): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 100.268609] LDISKFS-fs (vdd): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 104.158792] LDISKFS-fs (vde): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 108.054732] LDISKFS-fs (vdf): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 116.469790] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing load_modules_local [ 123.347888] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 123.392992] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 123.412733] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 124.560675] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 124.577086] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 124.625506] Lustre: lustre-MDT0000: new disk, initializing [ 124.690295] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 124.707373] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 126.902606] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 133.764138] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 133.813051] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 133.857811] Lustre: 6498:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 133.876389] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 133.878992] Lustre: Skipped 1 previous similar message [ 133.923271] Lustre: lustre-MDT0001: new disk, initializing [ 133.960672] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 133.985906] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 133.992452] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 135.836119] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 138.653760] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 143.531345] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 143.585778] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 143.741578] Lustre: lustre-OST0000: new disk, initializing [ 143.747556] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 143.788304] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 146.781917] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 150.514952] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 150.519511] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 150.556540] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 153.254855] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 153.329929] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 153.398952] Lustre: lustre-OST0001: new disk, initializing [ 153.405265] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 153.452279] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 156.389902] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 157.367094] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 157.371756] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 157.391266] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 164.266559] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 167.912562] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 175.695248] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing check_logdir /tmp/testlogs/ [ 178.065238] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing yml_node [ 180.538511] Lustre: DEBUG MARKER: Client: 2.16.59.50 [ 181.974842] Lustre: DEBUG MARKER: MDS: 2.16.59.50 [ 183.396705] Lustre: DEBUG MARKER: OSS: 2.16.59.50 [ 184.318898] Lustre: DEBUG MARKER: -----============= acceptance-small: conf-sanity ============----- Mon Nov 3 05:42:45 EST 2025 [ 193.321808] Lustre: DEBUG MARKER: excepting tests: 102 106 115 32newtarball [ 194.107036] Lustre: DEBUG MARKER: skipping tests SLOW=no: 45 69 106 111 114 [ 208.864146] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 208.864441] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 208.869103] Lustre: Skipped 2 previous similar messages [ 208.874457] Lustre: Skipped 3 previous similar messages [ 213.077331] LustreError: 12199:0:(obd_class.h:479:obd_check_dev()) Device 28 not setup [ 213.152152] Lustre: server umount lustre-MDT0000 complete [ 215.184055] LustreError: 6489:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1762166596 with bad export cookie 10345478453974894494 [ 215.186974] LustreError: MGC192.168.202.147@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 215.191850] LustreError: 6489:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 219.103943] LustreError: 6505:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 219.108164] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 219.108696] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 219.121193] LustreError: 6505:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 6 previous similar messages [ 219.129168] Lustre: Skipped 2 previous similar messages [ 220.128373] LustreError: 8159:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 220.128464] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 220.134524] LustreError: 8159:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 220.140989] Lustre: Skipped 2 previous similar messages [ 221.329227] LustreError: 12401:0:(obd_class.h:479:obd_check_dev()) Device 27 not setup [ 221.331184] LustreError: 12401:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 221.414809] Lustre: server umount lustre-MDT0001 complete [ 223.715391] LustreError: 12602:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 223.717422] LustreError: 12602:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 223.745140] Lustre: server umount lustre-OST0000 complete [ 225.703747] Lustre: server umount lustre-OST0001 complete [ 228.602854] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_hostid [ 233.449319] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing load_modules_local [ 238.426726] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 241.837516] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 244.923336] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 248.090869] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 251.916332] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 251.952991] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 252.081979] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 252.101237] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 252.151887] Lustre: lustre-MDT0000: new disk, initializing [ 252.209164] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 252.222039] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 254.312244] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 261.121938] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 261.165530] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 261.223829] Lustre: 14982:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 261.259490] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 261.262748] Lustre: Skipped 1 previous similar message [ 261.321075] Lustre: lustre-MDT0001: new disk, initializing [ 261.371617] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 261.408087] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 261.412394] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 263.344179] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 267.849770] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 269.493409] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 272.955582] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 272.985120] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 274.259884] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 274.324532] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 276.066676] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 280.122955] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 281.793588] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 281.935347] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 283.897302] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 284.006355] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 289.762092] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 289.762490] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 289.768407] LustreError: Skipped 1 previous similar message [ 289.775034] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 289.778671] Lustre: Skipped 1 previous similar message [ 291.066162] LustreError: 17922:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 291.072079] LustreError: 17922:0:(obd_class.h:479:obd_check_dev()) Skipped 9 previous similar messages [ 291.131694] Lustre: server umount lustre-OST0000 complete [ 294.883107] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 294.884868] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 294.888044] Lustre: Skipped 2 previous similar messages [ 299.225840] LustreError: 18123:0:(obd_class.h:479:obd_check_dev()) Device 22 not setup [ 299.228989] LustreError: 18123:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 299.278766] Lustre: server umount lustre-MDT0000 complete [ 300.000142] LustreError: 14993:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 300.005803] LustreError: 14993:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 301.159885] LustreError: 16729:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1762166682 with bad export cookie 10345478453974896034 [ 301.165625] LustreError: MGC192.168.202.147@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 301.165984] LustreError: 16729:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 301.364111] Lustre: server umount lustre-MDT0001 complete [ 303.645825] Lustre: DEBUG MARKER: == conf-sanity test 76a: set permanent params with lctl across mounts ========================================================== 05:44:44 (1762166684) [ 306.733158] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 306.988911] LustreError: 18849:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 307.039832] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 307.042769] Lustre: Skipped 1 previous similar message [ 308.968194] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 312.290269] LustreError: 18848:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 312.300533] LustreError: 18848:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 313.501557] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 315.686790] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 318.616804] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 320.550638] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 323.859356] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 324.003143] Lustre: lustre-OST0000: Not available for connect from 0@lo (not set up) [ 324.006380] Lustre: Skipped 1 previous similar message [ 327.120577] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 329.803977] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 337.308418] Lustre: Modifying parameter general.osc.*.max_dirty_mb=452 in log params [ 342.500217] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 342.503270] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 342.507449] Lustre: Skipped 1 previous similar message [ 346.770271] LustreError: 21152:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup [ 346.773448] LustreError: 21152:0:(obd_class.h:479:obd_check_dev()) Skipped 13 previous similar messages [ 346.834354] Lustre: server umount lustre-MDT0000 complete [ 347.616248] LustreError: 19911:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 347.623724] LustreError: 19911:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 348.577211] LustreError: 21046:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1762166729 with bad export cookie 10345478453974896916 [ 348.582229] LustreError: 21046:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 1 previous similar message [ 348.583795] LustreError: MGC192.168.202.147@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 348.731827] Lustre: server umount lustre-MDT0001 complete [ 360.937032] Lustre: server umount lustre-OST0000 complete [ 371.152794] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing load_modules_local [ 376.166604] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 376.382108] LustreError: 22882:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 376.422311] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 376.425410] Lustre: Skipped 2 previous similar messages [ 378.037941] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 382.017722] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 384.275781] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 385.859883] Lustre: Setting parameter general.debug_raw_pointers=Y in log params [ 389.607447] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 392.670349] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 396.936522] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 396.970720] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 397.043590] Lustre: lustre-OST0001: new disk, initializing [ 397.044930] Lustre: Skipped 1 previous similar message [ 397.048475] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 397.053224] Lustre: Skipped 1 previous similar message [ 398.430772] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 398.435362] Lustre: Skipped 1 previous similar message [ 398.437441] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 398.494789] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 399.948628] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 405.912750] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 407.728766] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 421.088827] LustreError: 26362:0:(obd_class.h:479:obd_check_dev()) Device 28 not setup [ 421.091492] LustreError: 26362:0:(obd_class.h:479:obd_check_dev()) Skipped 17 previous similar messages [ 421.135952] Lustre: server umount lustre-MDT0000 complete [ 422.370147] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 422.371450] LustreError: 22881:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 422.379380] Lustre: Skipped 3 previous similar messages [ 422.389728] LustreError: 22881:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 4 previous similar messages [ 422.836832] LustreError: 22861:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1762166804 with bad export cookie 10345478453974897763 [ 422.841398] LustreError: 22861:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 1 previous similar message [ 422.841965] LustreError: MGC192.168.202.147@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 425.076560] Lustre: server umount lustre-MDT0001 complete [ 436.747058] Lustre: server umount lustre-OST0000 complete [ 447.922034] Lustre: server umount lustre-OST0001 complete [ 455.336712] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing load_modules_local [ 459.534403] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 459.735812] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 459.738473] Lustre: Skipped 3 previous similar messages [ 461.201756] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 464.539244] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 466.355768] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 467.545862] Lustre: 29254:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 470.179768] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 472.577281] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 475.894179] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 478.258186] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 492.460249] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 493.857835] Lustre: 31129:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 505.827179] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 505.828507] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 505.830088] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 505.830092] Lustre: Skipped 2 previous similar messages [ 505.839910] Lustre: Skipped 4 previous similar messages [ 507.012991] LustreError: 31331:0:(obd_class.h:479:obd_check_dev()) Device 10 not setup [ 507.016342] LustreError: 31331:0:(obd_class.h:479:obd_check_dev()) Skipped 25 previous similar messages [ 507.056074] Lustre: server umount lustre-MDT0000 complete [ 508.530466] LustreError: 28119:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1762166889 with bad export cookie 10345478453974899058 [ 508.534210] LustreError: MGC192.168.202.147@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 508.535268] LustreError: 28119:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 508.666246] Lustre: server umount lustre-MDT0001 complete [ 520.202147] Lustre: server umount lustre-OST0000 complete [ 532.201892] Lustre: server umount lustre-OST0001 complete [ 536.067666] Lustre: DEBUG MARKER: == conf-sanity test 76b: verify params log setup correctly ========================================================== 05:48:37 (1762166917) [ 548.037728] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing load_modules_local [ 552.656678] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 552.889571] LustreError: 33912:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 552.899937] LustreError: 33912:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 554.846254] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 558.553492] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 560.181407] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 561.295040] Lustre: 35029:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 564.116034] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 566.597132] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 570.137066] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 572.609723] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 581.345553] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 582.938779] Lustre: 36881:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 593.895131] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 593.897589] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 593.900113] Lustre: Skipped 2 previous similar messages [ 593.903837] Lustre: Skipped 4 previous similar messages [ 596.108138] LustreError: 37033:0:(obd_class.h:479:obd_check_dev()) Device 10 not setup [ 596.111325] LustreError: 37033:0:(obd_class.h:479:obd_check_dev()) Skipped 25 previous similar messages [ 596.167182] Lustre: server umount lustre-MDT0000 complete [ 597.629270] LustreError: 34648:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1762166979 with bad export cookie 10345478453974900164 [ 597.632529] LustreError: MGC192.168.202.147@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 597.634082] LustreError: 34648:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 597.771785] Lustre: server umount lustre-MDT0001 complete [ 609.514561] Lustre: server umount lustre-OST0000 complete [ 621.630194] Lustre: server umount lustre-OST0001 complete [ 625.523648] Lustre: DEBUG MARKER: == conf-sanity test 76c: verify changelog_mask is applied with lctl set_param -P ========================================================== 05:50:06 (1762167006) [ 631.805145] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing load_modules_local [ 635.800302] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 636.023494] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 636.025782] Lustre: Skipped 7 previous similar messages [ 637.552872] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 641.380365] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 643.201778] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 644.554042] Lustre: 40121:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 647.268772] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 649.580679] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 652.859543] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 655.047196] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 668.984581] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 675.760927] Lustre: Setting parameter general.mdd.*.changelog_mask=-CLOSE in log params [ 675.762675] Lustre: Skipped 1 previous similar message [ 677.345158] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 677.348565] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 677.353715] Lustre: Skipped 1 previous similar message [ 683.050845] Lustre: server umount lustre-MDT0000 complete [ 684.517293] LustreError: 38984:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1762167065 with bad export cookie 10345478453974901270 [ 684.521199] LustreError: MGC192.168.202.147@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 684.522271] LustreError: 38984:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 686.675665] LustreError: 38999:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 192.168.202.47@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [ 686.681636] LustreError: 38999:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 5 previous similar messages [ 690.737497] Lustre: server umount lustre-MDT0001 complete [ 694.525464] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 709.600175] LustreError: 3643:0:(client.c:1380:ptlrpc_import_delay_req()) @@@ invalidate in flight req@ffff8c454170a680 x1847765462483584/t0(0) o250->MGC192.168.202.147@tcp@0@lo:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl Rpc:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 711.452122] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 714.939931] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 716.715238] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 719.022692] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 719.208751] Lustre: lustre-MDT0000-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 723.899252] Lustre: DEBUG MARKER: mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 4 sec [ 725.393664] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 728.165463] Lustre: DEBUG MARKER: mdc.lustre-MDT0001-mdc-*.mds_server_uuid in FULL state after 2 sec [ 730.593333] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 730.596658] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 730.598875] Lustre: Skipped 13 previous similar messages [ 735.947885] LustreError: 44395:0:(obd_class.h:479:obd_check_dev()) Device 10 not setup [ 735.951159] LustreError: 44395:0:(obd_class.h:479:obd_check_dev()) Skipped 41 previous similar messages [ 735.995710] Lustre: server umount lustre-MDT0000 complete [ 737.619541] LustreError: 42003:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1762167118 with bad export cookie 10345478453974902446 [ 737.622095] LustreError: MGC192.168.202.147@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 737.628160] LustreError: 42003:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 737.785154] Lustre: server umount lustre-MDT0001 complete [ 749.010313] Lustre: server umount lustre-OST0000 complete [ 759.723047] Lustre: server umount lustre-OST0001 complete [ 763.350368] Lustre: DEBUG MARKER: == conf-sanity test 76d: verify llite.*.xattr_cache can be set by 'lctl set_param -P' correctly ========================================================== 05:52:24 (1762167144) [ 769.797329] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing load_modules_local [ 774.157837] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 775.957705] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 779.331715] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 781.102653] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 782.443139] Lustre: 47484:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 782.447293] Lustre: 47484:0:(mgs_llog.c:1346:mgs_modify_param()) Skipped 1 previous similar message [ 784.881762] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 787.070491] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 790.262311] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 792.433814] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 806.427644] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 813.263632] Lustre: Setting parameter general.llite.*.xattr_cache=0 in log params [ 820.194523] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 820.200404] Lustre: Skipped 11 previous similar messages [ 824.925057] Lustre: server umount lustre-MDT0000 complete [ 826.249612] LustreError: 46345:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1762167207 with bad export cookie 10345478453974903230 [ 826.256295] LustreError: MGC192.168.202.147@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 826.396485] Lustre: server umount lustre-MDT0001 complete [ 838.660652] Lustre: server umount lustre-OST0000 complete [ 850.352778] Lustre: server umount lustre-OST0001 complete [ 854.143246] Lustre: DEBUG MARKER: == conf-sanity test 77: comma-separated MGS NIDs and failover node NIDs ========================================================== 05:53:55 (1762167235) [ 855.157338] Lustre: DEBUG MARKER: SKIP: conf-sanity test_77 mixed loopback and real device not working [ 855.864932] Lustre: DEBUG MARKER: == conf-sanity test 78: run resize2fs on MDT and OST filesystems ========================================================== 05:53:56 (1762167236) [ 860.157569] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_hostid [ 863.525573] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing load_modules_local [ 867.600088] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 870.066088] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 872.745891] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 872.778111] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 872.891614] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 872.932375] Lustre: lustre-MDT0000: new disk, initializing [ 872.973289] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 874.441914] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 877.773079] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 880.176025] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 880.207231] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 882.260260] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 882.273601] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 882.380079] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 885.645886] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 886.375069] Lustre: DEBUG MARKER: create test files [ 897.504646] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 903.643454] Lustre: server umount lustre-OST0000 complete [ 905.076767] Lustre: server umount lustre-MDT0000 complete [ 909.022868] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing unload_modules_local [ 910.218088] Key type lgssc unregistered [ 910.377403] LNet: 54931:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 910.379988] LNetError: 54931:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 910.389338] LNet: Removed LNI 192.168.202.147@tcp [ 910.757125] Key type .llcrypt unregistered [ 910.758634] Key type ._llcrypt unregistered [ 922.069205] Key type ._llcrypt registered [ 922.070130] Key type .llcrypt registered [ 922.110780] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing load_modules_local [ 922.532581] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 922.542128] alg: No test for adler32 (adler32-zlib) [ 923.402313] Lustre: Lustre: Build Version: 2.16.59_50_g4204c43 [ 923.493385] LNet: Added LNI 192.168.202.147@tcp [8/256/0/180] [ 925.087210] Key type lgssc registered [ 925.548672] Lustre: Echo OBD driver; http://www.lustre.org/ [ 928.434432] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 928.443466] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 929.614336] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 930.970181] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 932.864842] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 935.683222] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 935.817783] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 937.983616] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 938.918420] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:102 to 0x240000400:129) [ 938.953130] Lustre: 57512:0:(osd_internal.h:1471:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 515 < left 544, rollback = 9 [ 938.957111] Lustre: 57512:0:(osd_handler.c:2081:osd_trans_dump_creds()) create: 32/128/0, destroy: 0/0/0 [ 938.960888] Lustre: 57512:0:(osd_handler.c:2088:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 938.964021] Lustre: 57512:0:(osd_handler.c:2098:osd_trans_dump_creds()) write: 2/13/1, punch: 0/0/0, quota 1/3/0 [ 938.971065] Lustre: 57512:0:(osd_handler.c:2105:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 938.974704] Lustre: 57512:0:(osd_handler.c:2112:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 940.012352] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 943.059728] Lustre: DEBUG MARKER: check files after expanding the MDT and OST filesystems [ 944.759043] Lustre: DEBUG MARKER: create more files after expanding the MDT and OST filesystems [ 949.216817] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 949.216962] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 949.225626] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 952.879292] LustreError: 58254:0:(obd_class.h:479:obd_check_dev()) Device 13 not setup [ 952.922742] Lustre: server umount lustre-OST0000 complete [ 954.319225] LustreError: 58455:0:(obd_class.h:479:obd_check_dev()) Device 8 not setup [ 954.321244] LustreError: 58455:0:(obd_class.h:479:obd_check_dev()) Skipped 1 previous similar message [ 954.380605] Lustre: server umount lustre-MDT0000 complete [ 958.093294] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing unload_modules_local [ 959.248045] Key type lgssc unregistered [ 959.379586] LNet: 58987:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 959.383217] LNetError: 58987:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 959.391357] LNet: Removed LNI 192.168.202.147@tcp [ 959.717330] Key type .llcrypt unregistered [ 959.718857] Key type ._llcrypt unregistered [ 971.881615] Key type ._llcrypt registered [ 971.883209] Key type .llcrypt registered [ 971.926551] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing load_modules_local [ 972.301545] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 972.334564] alg: No test for adler32 (adler32-zlib) [ 973.192202] Lustre: Lustre: Build Version: 2.16.59_50_g4204c43 [ 973.277688] LNet: Added LNI 192.168.202.147@tcp [8/256/0/180] [ 974.855258] Key type lgssc registered [ 975.239921] Lustre: Echo OBD driver; http://www.lustre.org/ [ 978.089065] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 978.100854] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 979.294077] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 980.709461] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 982.660375] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 985.020284] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 985.128504] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 987.131225] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 989.016612] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 989.221950] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:140 to 0x240000400:161) [ 989.247917] Lustre: 61667:0:(osd_internal.h:1471:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 515 < left 544, rollback = 9 [ 989.251521] Lustre: 61667:0:(osd_handler.c:2081:osd_trans_dump_creds()) create: 32/128/0, destroy: 0/0/0 [ 989.254472] Lustre: 61667:0:(osd_handler.c:2088:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 989.257448] Lustre: 61667:0:(osd_handler.c:2098:osd_trans_dump_creds()) write: 2/13/1, punch: 0/0/0, quota 1/3/0 [ 989.260820] Lustre: 61667:0:(osd_handler.c:2105:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 989.263961] Lustre: 61667:0:(osd_handler.c:2112:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 997.288432] Lustre: DEBUG MARKER: check files after shrinking the MDT and OST filesystems [ 1001.647086] LustreError: 62361:0:(obd_class.h:479:obd_check_dev()) Device 13 not setup [ 1001.682622] Lustre: server umount lustre-OST0000 complete [ 1013.219284] LustreError: 62564:0:(obd_class.h:479:obd_check_dev()) Device 8 not setup [ 1013.221934] LustreError: 62564:0:(obd_class.h:479:obd_check_dev()) Skipped 1 previous similar message [ 1013.278949] Lustre: server umount lustre-MDT0000 complete [ 1017.147044] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing unload_modules_local [ 1018.293721] Key type lgssc unregistered [ 1018.431573] LNet: 63097:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1018.434850] LNetError: 63097:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1018.441558] LNet: Removed LNI 192.168.202.147@tcp [ 1018.740608] Key type .llcrypt unregistered [ 1018.742160] Key type ._llcrypt unregistered [ 1025.506352] Key type ._llcrypt registered [ 1025.508020] Key type .llcrypt registered [ 1025.568178] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_hostid [ 1032.657893] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing load_modules_local [ 1033.045704] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1033.082542] alg: No test for adler32 (adler32-zlib) [ 1033.983524] Lustre: Lustre: Build Version: 2.16.59_50_g4204c43 [ 1034.088336] LNet: Added LNI 192.168.202.147@tcp [8/256/0/180] [ 1035.687183] Key type lgssc registered [ 1036.156114] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1039.826164] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1042.601791] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1045.111930] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1047.600420] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1050.286784] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1050.308202] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1050.315489] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1051.406433] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 1051.418918] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1051.456042] Lustre: lustre-MDT0000: new disk, initializing [ 1051.484076] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1051.492150] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1052.872615] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 1057.278954] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1057.303311] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1057.339535] Lustre: 66195:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 1057.365591] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 1057.368196] Lustre: Skipped 1 previous similar message [ 1057.409113] Lustre: lustre-MDT0001: new disk, initializing [ 1057.444981] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 1057.460700] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 1057.465875] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 1058.882830] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 1062.179601] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1063.478553] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 1065.985315] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1066.008838] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1066.110857] Lustre: lustre-OST0000: new disk, initializing [ 1066.113665] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1066.145632] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1067.387548] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 1067.392114] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 1067.427913] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 1068.123058] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 1071.299728] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1072.806126] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 1072.892442] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 1074.356123] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 1074.438598] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 1077.170454] LustreError: 69132:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 1077.211887] Lustre: server umount lustre-OST0000 complete [ 1082.848626] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1082.849297] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1082.851865] Lustre: Skipped 1 previous similar message [ 1082.854856] Lustre: Skipped 1 previous similar message [ 1087.968862] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1087.970979] Lustre: Skipped 1 previous similar message [ 1087.972554] LustreError: 69332:0:(obd_class.h:479:obd_check_dev()) Device 22 not setup [ 1087.975460] LustreError: 69332:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 1088.000742] Lustre: server umount lustre-MDT0000 complete [ 1089.350344] LustreError: 66186:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1762167470 with bad export cookie 1029671795865306470 [ 1089.352847] LustreError: MGC192.168.202.147@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1089.354739] LustreError: 66186:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1098.723722] LustreError: 69536:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 1098.726273] LustreError: 69536:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 1098.775977] Lustre: server umount lustre-MDT0001 complete [ 1101.852342] Lustre: DEBUG MARKER: == conf-sanity test 79: format MDT/OST without mgs option (should return errors) ========================================================== 05:58:02 (1762167482) [ 1105.635938] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing load_modules_local [ 1110.425498] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1119.545112] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_hostid [ 1123.002850] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing load_modules_local [ 1126.712939] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1129.137219] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1131.519285] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1133.918092] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1136.542591] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1136.565894] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1136.662889] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 1136.678485] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1136.719883] Lustre: lustre-MDT0000: new disk, initializing [ 1136.752083] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1136.761834] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1138.323648] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 1143.078462] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1143.114725] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1143.156422] Lustre: 73768:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 1143.183516] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 1143.185635] Lustre: Skipped 1 previous similar message [ 1143.225474] Lustre: lustre-MDT0001: new disk, initializing [ 1143.271690] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 1143.291419] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 1143.295823] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 1144.882230] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 1148.291765] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1149.858552] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 1152.723232] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1152.752504] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1152.864236] Lustre: lustre-OST0000: new disk, initializing [ 1152.867160] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1152.901815] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1155.178368] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 1158.698212] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1159.169328] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 1159.172493] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 1159.219637] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 1160.216927] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 1160.301430] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 1161.753513] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 1161.839793] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 1164.257400] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1164.257995] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1164.262242] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1164.264765] LustreError: Skipped 1 previous similar message [ 1164.268490] Lustre: Skipped 1 previous similar message [ 1168.623912] LustreError: 76711:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 1168.626440] LustreError: 76711:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 1168.669664] Lustre: server umount lustre-OST0000 complete [ 1173.984413] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1173.988146] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1173.993428] Lustre: Skipped 1 previous similar message [ 1173.995705] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1176.191057] LustreError: 76913:0:(obd_class.h:479:obd_check_dev()) Device 22 not setup [ 1176.193067] LustreError: 76913:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 1176.221726] Lustre: server umount lustre-MDT0000 complete [ 1177.630861] LustreError: 74548:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1762167559 with bad export cookie 1029671795865307338 [ 1177.633049] LustreError: MGC192.168.202.147@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1177.636050] LustreError: 74548:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 1177.773914] Lustre: server umount lustre-MDT0001 complete [ 1180.883123] Lustre: DEBUG MARKER: == conf-sanity test 80: mgc import reconnect race ======== 05:59:22 (1762167562) [ 1183.459561] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1183.665593] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1185.110709] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 1188.139755] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1189.821402] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 1191.840860] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1193.262992] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 1195.857370] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1198.004201] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 1200.023856] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1201.303098] Lustre: 80121:0:(genops.c:1791:obd_export_evict_by_uuid()) MGS: evicting a8794a99-007c-4438-96eb-f604584c5e41 at adminstrative request [ 1203.169830] LustreError: MGC192.168.202.147@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1203.174337] Lustre: *** cfs_fail_loc=906, val=2147483648*** [ 1213.408566] Lustre: MGS: Client a8794a99-007c-4438-96eb-f604584c5e41 (at 0@lo) reconnecting [ 1213.412556] Lustre: Evicted from MGS (at 0@lo) after server handle changed from 0xe4a210afbca4c3c to 0xe4a210afbca4ec7 [ 1213.416669] Lustre: MGC192.168.202.147@tcp: Connection restored to 0@lo (at 0@lo) [ 1233.895579] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1233.926067] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1233.980912] Lustre: lustre-OST0001: new disk, initializing [ 1233.982732] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 1234.015305] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 1234.019072] Lustre: Skipped 2 previous similar messages [ 1236.109290] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 1239.276635] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid [ 1240.226065] LustreError: 81246:0:(obd_class.h:479:obd_check_dev()) Device 25 not setup [ 1240.227608] LustreError: 81246:0:(obd_class.h:479:obd_check_dev()) Skipped 13 previous similar messages [ 1240.241343] Lustre: server umount lustre-OST0001 complete [ 1240.572837] LustreError: 79383:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1240.579287] LustreError: 79383:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 1244.127806] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 1244.130230] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1244.133705] Lustre: Skipped 1 previous similar message [ 1244.135558] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1244.137484] Lustre: Skipped 2 previous similar messages [ 1245.666337] LustreError: 79383:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1245.673456] LustreError: 79383:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 1247.562939] Lustre: server umount lustre-OST0000 complete [ 1249.759907] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1249.762168] LustreError: Skipped 1 previous similar message [ 1249.763671] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1249.768054] Lustre: Skipped 1 previous similar message [ 1255.904090] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1255.906549] Lustre: Skipped 6 previous similar messages [ 1263.071113] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 1263.107337] LustreError: 81660:0:(obd_class.h:479:obd_check_dev()) Device 24 not setup [ 1263.109777] LustreError: 81660:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 1263.137845] Lustre: server umount lustre-MDT0000 complete [ 1264.440275] LustreError: 77765:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1762167645 with bad export cookie 1029671795865308871 [ 1264.443679] LustreError: MGC192.168.202.147@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1264.445410] LustreError: 77765:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1264.564478] Lustre: server umount lustre-MDT0001 complete [ 1267.459965] Lustre: DEBUG MARKER: == conf-sanity test 81: sparse OST indexing ============== 06:00:48 (1762167648) [ 1268.021458] Lustre: DEBUG MARKER: SKIP: conf-sanity test_81 needs >= 3 OSTs [ 1268.624473] Lustre: DEBUG MARKER: == conf-sanity test 82a: specify OSTs for file (succeed) or directory (succeed) ========================================================== 06:00:49 (1762167649) [ 1269.186910] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82a needs >= 3 OSTs [ 1269.778369] Lustre: DEBUG MARKER: == conf-sanity test 82b: specify OSTs for file with --pool and --ost-list options ========================================================== 06:00:50 (1762167650) [ 1270.318572] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82b needs >= 4 OSTs [ 1270.933773] Lustre: DEBUG MARKER: == conf-sanity test 83: ENOSPACE on OST doesn't cause message VFS: Busy inodes after unmount ... ========================================================== 06:00:52 (1762167652) [ 1271.461560] Lustre: DEBUG MARKER: mount the OST /dev/mapper/ost1_flakey as a ldiskfs filesystem [ 1273.045421] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1273.388825] loop: module loaded [ 1273.391826] loop0: detected capacity change from 0 to 4294967296 [ 1273.400880] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 1273.845591] Lustre: DEBUG MARKER: run llverfs in partial mode on the OST ldiskfs /mnt/lustre-ost1 [ 1274.738031] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing run_llverfs /mnt/lustre-ost1 -vpl no [ 1276.012718] Lustre: DEBUG MARKER: unmount the OST /dev/mapper/ost1_flakey [ 1278.826044] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1280.041205] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1281.250205] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1282.559133] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1284.793240] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1284.817865] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1284.829969] LustreError: 84445:0:(obd_config.c:783:class_setup()) setup lustre-OST0000-osd failed (-28) [ 1284.833079] LustreError: 84445:0:(obd_mount.c:193:lustre_start_simple()) lustre-OST0000-osd setup error -28 [ 1284.836149] LustreError: 84445:0:(tgt_mount.c:2447:server_fill_super()) Unable to start osd on /dev/mapper/ost1_flakey: -28 [ 1284.839529] LustreError: 84445:0:(super25.c:199:lustre_fill_super()) llite: Unable to mount : rc = -28 [ 1290.837354] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_hostid [ 1293.953018] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing load_modules_local [ 1297.429082] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1299.646675] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1301.760435] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1303.856845] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1305.964325] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1305.988820] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1306.064861] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 1306.074503] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1306.109884] Lustre: lustre-MDT0000: new disk, initializing [ 1306.133175] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1306.138698] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1307.361469] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 1311.455484] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1311.480065] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1311.516968] Lustre: 87282:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 1311.622574] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 1312.924495] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 1316.116842] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1317.516432] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 1319.750940] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1319.776471] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1321.718425] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 1324.862572] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1326.232898] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 1327.606991] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 1327.636840] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 1328.352783] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 2 sec [ 1329.646129] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 1329.721120] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 1332.704804] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1332.706180] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1332.707682] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1332.711577] Lustre: Skipped 2 previous similar messages [ 1332.713197] Lustre: Skipped 2 previous similar messages [ 1336.433112] LustreError: 90235:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 1336.435715] LustreError: 90235:0:(obd_class.h:479:obd_check_dev()) Skipped 17 previous similar messages [ 1336.478618] Lustre: server umount lustre-OST0000 complete [ 1342.431830] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1342.434723] LustreError: Skipped 1 previous similar message [ 1352.159132] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 1352.215596] Lustre: server umount lustre-MDT0000 complete [ 1353.183899] LustreError: 88070:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1353.189779] LustreError: 88070:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 1353.473291] LustreError: 87273:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1762167734 with bad export cookie 1029671795865309459 [ 1353.476399] LustreError: MGC192.168.202.147@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1353.477076] LustreError: 87273:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1353.589318] Lustre: server umount lustre-MDT0001 complete [ 1356.410413] Lustre: DEBUG MARKER: == conf-sanity test 84: check recovery_hard_time ========= 06:02:17 (1762167737) [ 1358.601328] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1358.741253] LustreError: 91309:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1360.025217] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 1363.053766] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1364.426667] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 1366.192190] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1367.350265] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 1369.423929] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1371.233886] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 1372.945840] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1374.203242] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 1374.274566] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 1375.577113] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 1375.655066] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 1377.912768] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1377.935743] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1377.987952] Lustre: lustre-OST0001: new disk, initializing [ 1377.989488] Lustre: Skipped 2 previous similar messages [ 1377.991788] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 1377.993202] Lustre: Skipped 3 previous similar messages [ 1379.787833] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 1381.056782] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 1381.059838] Lustre: Skipped 2 previous similar messages [ 1381.061970] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 1381.098194] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 1382.852443] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid [ 1384.142930] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid 50 [ 1384.213816] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 1385.472262] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [ 1385.551454] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 1395.192278] Lustre: DEBUG MARKER: mds1 REPLAY BARRIER on lustre-MDT0000 [ 1400.320513] Lustre: Failing over lustre-MDT0000 [ 1400.579973] LustreError: 96190:0:(obd_class.h:479:obd_check_dev()) Device 28 not setup [ 1400.581886] LustreError: 96190:0:(obd_class.h:479:obd_check_dev()) Skipped 17 previous similar messages [ 1400.615535] Lustre: server umount lustre-MDT0000 complete [ 1401.825530] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1401.825530] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1401.825540] Lustre: Skipped 2 previous similar messages [ 1401.829716] Lustre: Skipped 2 previous similar messages [ 1401.834126] LustreError: 96307:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1401.840759] LustreError: 96307:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 1412.064875] LustreError: 92733:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1412.071162] LustreError: 92733:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 11 previous similar messages [ 1415.022979] LDISKFS-fs (dm-0): 4 truncates cleaned up [ 1415.024236] LDISKFS-fs (dm-0): recovery complete [ 1415.027021] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1415.058877] LustreError: MGC192.168.202.147@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1415.157730] Lustre: lustre-MDT0000: in recovery but waiting for the first client to connect [ 1416.312208] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 1417.805514] Lustre: lustre-MDT0000: Will be in recovery for at least 1:00, or until 3 clients reconnect [ 1417.806067] Lustre: 96307:0:(ldlm_lib.c:2067:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 1420.257277] Lustre: 91309:0:(ldlm_lib.c:2067:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 1420.259499] Lustre: 91309:0:(ldlm_lib.c:2067:extend_recovery_timer()) Skipped 1 previous similar message [ 1420.263124] Lustre: lustre-MDT0000-lwp-OST0001: Connection restored to 0@lo (at 0@lo) [ 1420.283657] LustreError: 96939:0:(ldlm_lib.c:2679:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 1420.599113] LustreError: 96939:0:(ldlm_lib.c:2679:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 1420.913259] LustreError: 96939:0:(ldlm_lib.c:2679:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 1420.916125] LustreError: 96939:0:(ldlm_lib.c:2679:replay_request_or_update()) Skipped 1 previous similar message [ 1421.231104] LustreError: 96939:0:(ldlm_lib.c:2679:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 1421.233389] LustreError: 96939:0:(ldlm_lib.c:2679:replay_request_or_update()) Skipped 1 previous similar message [ 1421.553259] Lustre: 96939:0:(ldlm_lib.c:2067:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 1421.557125] Lustre: 96939:0:(ldlm_lib.c:2067:extend_recovery_timer()) Skipped 8 previous similar messages [ 1422.185355] LustreError: 96939:0:(ldlm_lib.c:2679:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 1422.188694] LustreError: 96939:0:(ldlm_lib.c:2679:replay_request_or_update()) Skipped 3 previous similar messages [ 1422.503088] LustreError: 96939:0:(ldlm_lib.c:2679:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 1422.505301] LustreError: 96939:0:(ldlm_lib.c:2679:replay_request_or_update()) Skipped 3 previous similar messages [ 1423.759920] Lustre: 96939:0:(ldlm_lib.c:2067:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 1423.764213] Lustre: 96939:0:(ldlm_lib.c:2067:extend_recovery_timer()) Skipped 6 previous similar messages [ 1424.391983] LustreError: 96939:0:(ldlm_lib.c:2679:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 1424.395357] LustreError: 96939:0:(ldlm_lib.c:2679:replay_request_or_update()) Skipped 6 previous similar messages [ 1424.711100] LustreError: 96939:0:(ldlm_lib.c:2679:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 1424.713407] LustreError: 96939:0:(ldlm_lib.c:2679:replay_request_or_update()) Skipped 6 previous similar messages [ 1427.849495] Lustre: 96939:0:(ldlm_lib.c:2067:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 1427.854396] Lustre: 96939:0:(ldlm_lib.c:2067:extend_recovery_timer()) Skipped 12 previous similar messages [ 1428.481449] LustreError: 96939:0:(ldlm_lib.c:2679:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 1428.485752] LustreError: 96939:0:(ldlm_lib.c:2679:replay_request_or_update()) Skipped 12 previous similar messages [ 1428.799170] LustreError: 96939:0:(ldlm_lib.c:2679:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 1428.801353] LustreError: 96939:0:(ldlm_lib.c:2679:replay_request_or_update()) Skipped 12 previous similar messages [ 1435.985229] Lustre: 96939:0:(ldlm_lib.c:2067:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 1435.988308] Lustre: 96939:0:(ldlm_lib.c:2067:extend_recovery_timer()) Skipped 25 previous similar messages [ 1436.617458] LustreError: 96939:0:(ldlm_lib.c:2679:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 1436.621213] LustreError: 96939:0:(ldlm_lib.c:2679:replay_request_or_update()) Skipped 25 previous similar messages [ 1436.935116] LustreError: 96939:0:(ldlm_lib.c:2679:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 1436.939136] LustreError: 96939:0:(ldlm_lib.c:2679:replay_request_or_update()) Skipped 25 previous similar messages [ 1452.233274] Lustre: 96939:0:(ldlm_lib.c:2067:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 1452.236856] Lustre: 96939:0:(ldlm_lib.c:2067:extend_recovery_timer()) Skipped 51 previous similar messages [ 1452.865968] LustreError: 96939:0:(ldlm_lib.c:2679:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 1452.870342] LustreError: 96939:0:(ldlm_lib.c:2679:replay_request_or_update()) Skipped 51 previous similar messages [ 1453.183125] LustreError: 96939:0:(ldlm_lib.c:2679:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 1453.186116] LustreError: 96939:0:(ldlm_lib.c:2679:replay_request_or_update()) Skipped 51 previous similar messages [ 1477.215178] Lustre: 96939:0:(ldlm_lib.c:2375:target_recovery_overseer()) lustre-MDT0000 recovery is aborted by hard timeout [ 1477.217493] Lustre: 96939:0:(ldlm_lib.c:2385:target_recovery_overseer()) recovery is aborted, evict exports in recovery [ 1477.219664] Lustre: 96939:0:(genops.c:1618:class_disconnect_stale_exports()) lustre-MDT0000: disconnect stale client df22bf0e-cff8-471c-861c-510305d86287@192.168.202.47@tcp [ 1477.223259] Lustre: lustre-MDT0000: disconnecting 1 stale clients [ 1477.225578] Lustre: 96939:0:(ldlm_lib.c:1897:abort_req_replay_queue()) @@@ aborted: req@00000000130272fd x1847766452079232/t0(8589934779) o101->df22bf0e-cff8-471c-861c-510305d86287@192.168.202.47@tcp:134/0 lens 592/0 e 0 to 0 dl 1762167869 ref 1 fl Complete:/604/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:0 [ 1477.234167] Lustre: lustre-MDT0000-osd: cancel update llog [0x200000400:0x1:0x0] [ 1477.237091] Lustre: lustre-MDT0000: Denying connection for new client df22bf0e-cff8-471c-861c-510305d86287 (at 192.168.202.47@tcp), waiting for 3 known clients (2 recovered, 0 in progress, and 1 evicted) already passed deadline 0:00 [ 1477.241586] Lustre: lustre-MDT0001-osp-MDT0000: cancel update llog [0x240000401:0x1:0x0] [ 1477.255753] Lustre: lustre-MDT0000-osp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 1477.255926] Lustre: lustre-MDT0000: Recovery over after 1:00, of 3 clients 2 recovered and 1 was evicted. [ 1477.259793] Lustre: Skipped 2 previous similar messages [ 1477.280154] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:47 to 0x2c0000401:545) [ 1477.280173] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:80 to 0x280000401:577) [ 1481.699823] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1481.705478] Lustre: Skipped 1 previous similar message [ 1481.708150] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1481.709820] Lustre: Skipped 5 previous similar messages [ 1483.450769] Lustre: server umount lustre-OST0000 complete [ 1486.304114] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 1486.308704] LustreError: 92906:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1486.314711] LustreError: 92906:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 1499.103147] Lustre: lustre-OST0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 1499.160722] Lustre: server umount lustre-OST0001 complete [ 1506.662243] Lustre: server umount lustre-MDT0000 complete [ 1507.868246] LustreError: 91289:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1762167889 with bad export cookie 1029671795865360629 [ 1507.870223] LustreError: MGC192.168.202.147@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1507.873627] LustreError: 91289:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 1 previous similar message [ 1507.986923] Lustre: server umount lustre-MDT0001 complete [ 1510.771632] Lustre: DEBUG MARKER: == conf-sanity test 85: osd_ost init: fail ea_fid_set ==== 06:04:51 (1762167891) [ 1513.191578] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1513.195180] Lustre: *** cfs_fail_loc=197, val=0*** [ 1528.607615] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 1533.727383] LustreError: 98892:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.202.147@tcp: failed processing log, type 4: rc = -110 [ 1538.855259] Lustre: *** cfs_fail_loc=197, val=0*** [ 1538.856386] Lustre: Skipped 65 previous similar messages [ 1559.327386] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1559.329446] Lustre: Skipped 7 previous similar messages [ 1561.211717] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 1562.907510] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1563.535125] LustreError: 99556:0:(obd_class.h:479:obd_check_dev()) Device 1 not setup [ 1563.537657] LustreError: 99556:0:(obd_class.h:479:obd_check_dev()) Skipped 33 previous similar messages [ 1563.574835] Lustre: server umount lustre-OST0000 complete [ 1566.247918] Lustre: DEBUG MARKER: == conf-sanity test 86: Replacing mkfs.lustre -G option == 06:05:47 (1762167947) [ 1579.830160] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1579.979303] LustreError: 100398:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1579.987465] LustreError: 100398:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 7 previous similar messages [ 1581.293236] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 1584.415990] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1586.330512] LustreError: 100439:0:(osp_object.c:617:osp_attr_get()) lustre-MDT0001-osp-MDT0000: osp_attr_get update error [0x200000009:0x1:0x0]: rc = -5 [ 1586.335546] LustreError: 100439:0:(lod_sub_object.c:917:lod_sub_prep_llog()) lustre-MDT0000-mdtlov: can't get id from catalogs: rc = -5 [ 1586.339840] LustreError: 100439:0:(lod_dev.c:510:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 6, retries 0, failed: rc = -5 [ 1586.445696] Lustre: server umount lustre-MDT0000 complete [ 1591.317165] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1596.425896] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_hostid [ 1599.267839] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing load_modules_local [ 1602.673421] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1604.905374] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1606.986443] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1609.200670] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1611.644283] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1611.669216] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1611.768380] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 1611.784159] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1611.826322] Lustre: lustre-MDT0000: new disk, initializing [ 1611.862781] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1613.298641] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 1617.489865] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1617.509120] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1617.542722] Lustre: 104469:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 1617.647255] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 1618.845120] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 1621.800616] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1622.902914] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 1625.056741] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1625.077808] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1626.144490] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 1627.029633] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 1629.996809] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1631.273662] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 1631.350185] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 1632.599534] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 1632.661826] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 1636.320820] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1636.322771] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1636.323665] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1636.327114] Lustre: Skipped 6 previous similar messages [ 1636.329208] Lustre: Skipped 8 previous similar messages [ 1639.372907] Lustre: server umount lustre-OST0000 complete [ 1646.908683] Lustre: server umount lustre-MDT0000 complete [ 1649.961421] LustreError: 105252:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1762168031 with bad export cookie 1029671795865371290 [ 1649.972786] LustreError: MGC192.168.202.147@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1649.976958] LustreError: 105252:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 1 previous similar message [ 1650.435794] Lustre: server umount lustre-MDT0001 complete [ 1660.329282] Lustre: DEBUG MARKER: == conf-sanity test 87: check if MDT inode can hold EAs with N stripes properly ========================================================== 06:07:20 (1762168040) [ 1669.528242] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing unload_modules_local [ 1672.820521] Key type lgssc unregistered [ 1673.223890] LNet: 108539:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1673.269338] LNetError: 108539:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1674.348239] LNet: Removed LNI 192.168.202.147@tcp [ 1675.470140] Key type .llcrypt unregistered [ 1675.481075] Key type ._llcrypt unregistered [ 1691.298467] Key type ._llcrypt registered [ 1691.326289] Key type .llcrypt registered [ 1691.403920] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_hostid [ 1700.856941] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing load_modules_local [ 1701.224163] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1701.244340] alg: No test for adler32 (adler32-zlib) [ 1702.152144] Lustre: Lustre: Build Version: 2.16.59_50_g4204c43 [ 1702.289314] LNet: Added LNI 192.168.202.147@tcp [8/256/0/180] [ 1703.887482] Key type lgssc registered [ 1704.382708] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1708.165534] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1710.768300] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1713.277459] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1716.029948] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1718.348711] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1721.318035] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1721.352089] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1721.359860] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1722.477688] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 1722.496275] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1722.539828] Lustre: lustre-MDT0000: new disk, initializing [ 1722.585784] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1722.598055] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1724.316270] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 1728.884281] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1728.909442] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1729.016141] Lustre: lustre-OST0000: new disk, initializing [ 1729.018767] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1729.021315] Lustre: Skipped 1 previous similar message [ 1729.054145] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1730.635151] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 1730.640296] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 1730.652734] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 1731.475135] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 1736.337341] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1736.375198] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1736.440375] Lustre: lustre-OST0001: new disk, initializing [ 1736.442809] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 1736.473589] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 1737.695756] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:1:ost [ 1737.699981] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:1:ost] [ 1737.720636] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x280000400 [ 1738.970892] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 1747.937487] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1747.939054] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1747.946505] Lustre: Skipped 1 previous similar message [ 1747.952609] Lustre: Skipped 1 previous similar message [ 1749.980126] LustreError: 114091:0:(obd_class.h:479:obd_check_dev()) Device 17 not setup [ 1750.086968] Lustre: server umount lustre-MDT0000 complete [ 1752.583399] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: (null) [ 1754.414908] LustreError: 113496:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1762168135 with bad export cookie 17228756971645423517 [ 1754.418191] LustreError: MGC192.168.202.147@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1754.419843] LustreError: 113496:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 1 previous similar message [ 1754.451272] LustreError: 114685:0:(obd_class.h:479:obd_check_dev()) Device 12 not setup [ 1754.454275] LustreError: 114685:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 1754.482277] Lustre: server umount lustre-OST0000 complete [ 1756.217579] LustreError: 114886:0:(obd_class.h:479:obd_check_dev()) Device 2 not setup [ 1756.220499] LustreError: 114886:0:(obd_class.h:479:obd_check_dev()) Skipped 1 previous similar message [ 1756.276874] Lustre: server umount lustre-OST0001 complete [ 1760.053190] Lustre: DEBUG MARKER: == conf-sanity test 88: check the default mount options can be overridden ========================================================== 06:09:00 (1762168140) [ 1764.079628] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing load_modules_local [ 1767.921960] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1768.326116] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1770.625224] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1771.016368] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1773.809693] Lustre: DEBUG MARKER: == conf-sanity test 89: check tunefs --param and --erase-param{s} options ========================================================== 06:09:14 (1762168154) [ 1779.487784] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1780.245228] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1781.021215] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1781.851506] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1782.234542] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1782.955336] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1789.263404] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_hostid [ 1793.538462] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing load_modules_local [ 1797.448289] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1799.829199] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1802.269776] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1805.512210] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1809.095056] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1809.129991] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1809.255923] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 1809.280382] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1809.328558] Lustre: lustre-MDT0000: new disk, initializing [ 1809.383797] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1809.399187] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1811.148485] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 1816.624432] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1816.663771] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1816.710591] Lustre: 120200:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 1816.740359] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 1816.742783] Lustre: Skipped 1 previous similar message [ 1816.783431] Lustre: lustre-MDT0001: new disk, initializing [ 1816.828797] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 1816.847050] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 1816.851253] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 1818.582611] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 1822.627451] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1824.968888] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 1829.467828] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 1829.527638] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1829.716322] Lustre: lustre-OST0000: new disk, initializing [ 1829.718781] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1829.784947] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1833.381458] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 1838.407862] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount (FULL|IDLE) failover.node=192.0.2.254@tcp0,osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1839.646206] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 1839.650554] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 1839.717350] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 1840.781508] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 1840.896437] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 1843.128342] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 1843.260212] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 1844.710658] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1844.713140] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1844.718221] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1844.722967] Lustre: Skipped 1 previous similar message [ 1846.394091] LustreError: 123135:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 1846.396694] LustreError: 123135:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 1846.486695] Lustre: server umount lustre-OST0000 complete [ 1849.825975] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1849.831898] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1849.834534] Lustre: Skipped 1 previous similar message [ 1854.673750] LustreError: 123337:0:(obd_class.h:479:obd_check_dev()) Device 22 not setup [ 1854.675805] LustreError: 123337:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 1854.727342] Lustre: server umount lustre-MDT0000 complete [ 1856.798897] LustreError: 120192:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1762168238 with bad export cookie 17228756971645424728 [ 1856.802581] LustreError: MGC192.168.202.147@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1856.806792] LustreError: 120192:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1856.974935] Lustre: server umount lustre-MDT0001 complete [ 1861.570573] Lustre: DEBUG MARKER: == conf-sanity test 90a: check max_mod_rpcs_in_flight is enforced ========================================================== 06:10:42 (1762168242) [ 1865.046797] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1865.329215] LustreError: 124209:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1865.338912] LustreError: 124209:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 1865.378069] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1867.243536] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 1870.816397] LustreError: 124209:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1871.894091] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1874.079919] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 1876.958644] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1879.013405] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 1882.828442] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 1886.305808] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 1889.411780] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1901.408761] Lustre: *** cfs_fail_loc=159, val=0*** [ 1918.050839] Lustre: lustre-MDT0000: Client 38cfd924-cfb6-4785-9c89-91462d395c4a (at 192.168.202.47@tcp) reconnecting [ 1918.738808] Lustre: *** cfs_fail_loc=159, val=0*** [ 1918.744274] Lustre: Skipped 5 previous similar messages [ 1934.959615] Lustre: lustre-MDT0000: Client 38cfd924-cfb6-4785-9c89-91462d395c4a (at 192.168.202.47@tcp) reconnecting [ 1951.713097] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1951.715693] LustreError: Skipped 1 previous similar message [ 1951.717657] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1951.722980] Lustre: Skipped 1 previous similar message [ 1951.727358] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1953.765283] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 1956.979247] LustreError: 126719:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 1957.001066] LustreError: 126719:0:(obd_class.h:479:obd_check_dev()) Skipped 13 previous similar messages [ 1957.049953] Lustre: server umount lustre-OST0000 complete [ 1959.393535] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1959.398543] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1959.403814] Lustre: Skipped 1 previous similar message [ 1959.412863] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1959.415650] Lustre: Skipped 2 previous similar messages [ 1965.268566] Lustre: server umount lustre-MDT0000 complete [ 1967.462316] LustreError: 124188:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1762168348 with bad export cookie 17228756971645425596 [ 1967.469628] LustreError: MGC192.168.202.147@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1967.475391] LustreError: 124188:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 1967.702241] Lustre: server umount lustre-MDT0001 complete [ 1974.041251] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing unload_modules_local [ 1975.521818] Key type lgssc unregistered [ 1975.688969] LNet: 127655:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1975.696025] LNetError: 127655:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1975.704893] LNet: Removed LNI 192.168.202.147@tcp [ 1976.097737] Key type .llcrypt unregistered [ 1976.099242] Key type ._llcrypt unregistered [ 1988.958020] Key type ._llcrypt registered [ 1988.960262] Key type .llcrypt registered [ 1989.040675] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing load_modules_local [ 1989.647380] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1989.668543] alg: No test for adler32 (adler32-zlib) [ 1990.603135] Lustre: Lustre: Build Version: 2.16.59_50_g4204c43 [ 1990.726905] LNet: Added LNI 192.168.202.147@tcp [8/256/0/180] [ 1992.343173] Key type lgssc registered [ 1992.990144] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1997.881640] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1997.901813] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1999.229503] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2001.403411] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 2004.452356] LustreError: 129029:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2004.463827] LustreError: 129029:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 2007.142053] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2007.404488] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2010.093640] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 2013.527491] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2015.688357] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2019.312760] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2019.478527] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2022.021397] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 2024.229225] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2024.616943] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:42 to 0x280000401:65) [ 2039.340291] Lustre: *** cfs_fail_loc=159, val=0*** [ 2054.754601] Lustre: lustre-MDT0000: Client ae5f8915-c671-49fd-8c95-65252522e0a3 (at 192.168.202.47@tcp) reconnecting [ 2054.855626] Lustre: 129024:0:(osd_internal.h:1471:osd_trans_exec_op()) lustre-MDT0001: opcode 2: before 507 < left 525, rollback = 2 [ 2054.861063] Lustre: 129024:0:(osd_handler.c:2081:osd_trans_dump_creds()) create: 3/12/6, destroy: 0/0/0 [ 2054.866782] Lustre: 129024:0:(osd_handler.c:2088:osd_trans_dump_creds()) attr_set: 3/3/0, xattr_set: 6/525/0 [ 2054.870951] Lustre: 129024:0:(osd_handler.c:2098:osd_trans_dump_creds()) write: 7/153/0, punch: 0/0/0, quota 1/3/0 [ 2054.876256] Lustre: 129024:0:(osd_handler.c:2105:osd_trans_dump_creds()) insert: 7/135/3, delete: 0/0/0 [ 2054.880768] Lustre: 129024:0:(osd_handler.c:2112:osd_trans_dump_creds()) ref_add: 1/1/0, ref_del: 0/0/0 [ 2055.817807] Lustre: *** cfs_fail_loc=159, val=0*** [ 2072.163776] Lustre: lustre-MDT0001: Client ae5f8915-c671-49fd-8c95-65252522e0a3 (at 192.168.202.47@tcp) reconnecting [ 2072.796337] Lustre: *** cfs_fail_loc=159, val=0*** [ 2072.798268] Lustre: Skipped 3 previous similar messages [ 2089.061178] Lustre: lustre-MDT0001: Client ae5f8915-c671-49fd-8c95-65252522e0a3 (at 192.168.202.47@tcp) reconnecting [ 2090.560852] Lustre: 131751:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 2091.649180] Lustre: *** cfs_fail_loc=159, val=0*** [ 2091.651321] Lustre: Skipped 4 previous similar messages [ 2108.015616] Lustre: lustre-MDT0000: Client 00d0d673-c5ac-46fe-a408-1b2c64e7237d (at 192.168.202.47@tcp) reconnecting [ 2108.813552] Lustre: *** cfs_fail_loc=159, val=0*** [ 2108.818737] Lustre: Skipped 13 previous similar messages [ 2125.409555] Lustre: lustre-MDT0000: Client 00d0d673-c5ac-46fe-a408-1b2c64e7237d (at 192.168.202.47@tcp) reconnecting [ 2126.053835] Lustre: 132043:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 2142.692082] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2142.694830] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2142.704716] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2144.738960] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2144.739709] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2144.741736] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2148.658109] LustreError: 132168:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 2148.705375] Lustre: server umount lustre-OST0000 complete [ 2150.880791] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2150.886297] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2150.895394] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2150.897871] Lustre: Skipped 1 previous similar message [ 2154.978649] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2154.979218] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2156.491983] LustreError: 132369:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup [ 2156.494624] LustreError: 132369:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 2156.561448] Lustre: server umount lustre-MDT0000 complete [ 2158.296092] LustreError: 131650:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1762168539 with bad export cookie 7032706456914824524 [ 2158.301109] LustreError: MGC192.168.202.147@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2158.301435] LustreError: 131650:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 2158.395000] LustreError: 132571:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [ 2158.398473] LustreError: 132571:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 2158.501389] Lustre: server umount lustre-MDT0001 complete [ 2163.171578] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing unload_modules_local [ 2164.492426] Key type lgssc unregistered [ 2164.647978] LNet: 133102:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2164.654220] LNetError: 133102:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2164.661720] LNet: Removed LNI 192.168.202.147@tcp [ 2165.079319] Key type .llcrypt unregistered [ 2165.080934] Key type ._llcrypt unregistered [ 2176.564992] Key type ._llcrypt registered [ 2176.566433] Key type .llcrypt registered [ 2176.618256] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing load_modules_local [ 2177.029773] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2177.037588] alg: No test for adler32 (adler32-zlib) [ 2177.923083] Lustre: Lustre: Build Version: 2.16.59_50_g4204c43 [ 2178.037549] LNet: Added LNI 192.168.202.147@tcp [8/256/0/180] [ 2179.639341] Key type lgssc registered [ 2180.161365] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2183.709927] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 2183.722336] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2184.980732] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2186.541605] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 2189.927934] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2190.066502] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2191.567349] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 2193.597319] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2195.131461] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2198.124838] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2198.267491] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2200.295888] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:97) [ 2200.602364] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 2202.988247] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2210.538234] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:129) [ 2216.189057] Lustre: 136743:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 2217.234669] Lustre: 136791:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 2217.978654] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 in log params [ 2218.503725] Lustre: Setting parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=8 in log params [ 2219.890994] Lustre: 136938:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify lustre/mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 (mode = 0) failed: rc = -17 [ 2220.301145] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=7 in log params [ 2220.700231] Lustre: 137036:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 2226.145887] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2226.146838] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2226.148633] LustreError: Skipped 1 previous similar message [ 2226.149285] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2226.153012] Lustre: Skipped 1 previous similar message [ 2226.158702] Lustre: Skipped 1 previous similar message [ 2231.266847] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2231.270183] Lustre: Skipped 1 previous similar message [ 2236.383294] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2236.385425] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2236.389637] Lustre: Skipped 1 previous similar message [ 2236.406401] LustreError: 137133:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 2236.451746] Lustre: server umount lustre-OST0000 complete [ 2241.506165] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2241.506232] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2241.507182] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2241.511808] Lustre: Skipped 1 previous similar message [ 2246.624697] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2246.631039] Lustre: Skipped 2 previous similar messages [ 2252.255162] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2252.293218] LustreError: 137342:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup [ 2252.296115] LustreError: 137342:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 2252.336361] Lustre: server umount lustre-MDT0000 complete [ 2253.814679] LustreError: 134456:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1762168635 with bad export cookie 13379599696709567386 [ 2253.818712] LustreError: MGC192.168.202.147@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2253.820256] LustreError: 134456:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 2253.900851] LustreError: 137545:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [ 2253.903634] LustreError: 137545:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 2253.965364] Lustre: server umount lustre-MDT0001 complete [ 2258.160177] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing unload_modules_local [ 2259.375283] Key type lgssc unregistered [ 2259.513435] LNet: 138076:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2259.517036] LNetError: 138076:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2259.526446] LNet: Removed LNI 192.168.202.147@tcp [ 2259.868317] Key type .llcrypt unregistered [ 2259.869693] Key type ._llcrypt unregistered [ 2269.811498] Key type ._llcrypt registered [ 2269.812622] Key type .llcrypt registered [ 2269.857895] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing load_modules_local [ 2270.203840] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2270.234092] alg: No test for adler32 (adler32-zlib) [ 2271.101047] Lustre: Lustre: Build Version: 2.16.59_50_g4204c43 [ 2271.190678] LNet: Added LNI 192.168.202.147@tcp [8/256/0/180] [ 2272.783171] Key type lgssc registered [ 2273.218657] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2275.949372] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 2275.955418] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2277.120928] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2278.719927] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 2282.213239] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2282.384708] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2283.876632] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 2285.892240] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2287.213204] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2289.692381] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2289.854607] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2291.911598] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 2291.948523] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:129) [ 2293.874586] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2302.182868] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:161) [ 2306.886991] Lustre: *** cfs_fail_loc=159, val=0*** [ 2322.527814] Lustre: lustre-MDT0000: Client ca44d951-49cb-4c4b-9aad-a09140cf300b (at 192.168.202.47@tcp) reconnecting [ 2328.034748] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2328.037201] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2328.038789] LustreError: Skipped 1 previous similar message [ 2328.045816] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2328.048726] Lustre: Skipped 1 previous similar message [ 2330.228077] LustreError: 141844:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 2330.273678] Lustre: server umount lustre-OST0000 complete [ 2333.152423] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2333.153480] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2333.157784] Lustre: Skipped 2 previous similar messages [ 2338.046437] LustreError: 142045:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup [ 2338.048925] LustreError: 142045:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 2338.080052] Lustre: server umount lustre-MDT0000 complete [ 2338.272845] LustreError: 139450:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2338.277402] LustreError: 139450:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 2339.527524] LustreError: 141076:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1762168720 with bad export cookie 16878834577175367643 [ 2339.530170] LustreError: MGC192.168.202.147@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2339.531709] LustreError: 141076:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 2339.614133] LustreError: 142248:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [ 2339.617137] LustreError: 142248:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 2339.695805] Lustre: server umount lustre-MDT0001 complete [ 2343.925248] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing unload_modules_local [ 2345.211236] Key type lgssc unregistered [ 2345.376466] LNet: 142779:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2345.379447] LNetError: 142779:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2345.386476] LNet: Removed LNI 192.168.202.147@tcp [ 2345.758335] Key type .llcrypt unregistered [ 2345.760429] Key type ._llcrypt unregistered [ 2356.592365] Key type ._llcrypt registered [ 2356.593687] Key type .llcrypt registered [ 2356.635606] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing load_modules_local [ 2357.056564] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2357.063927] alg: No test for adler32 (adler32-zlib) [ 2357.934017] Lustre: Lustre: Build Version: 2.16.59_50_g4204c43 [ 2358.029825] LNet: Added LNI 192.168.202.147@tcp [8/256/0/180] [ 2359.615269] Key type lgssc registered [ 2360.063321] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2362.987543] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 2362.996417] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2364.176543] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2365.648964] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 2368.926847] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2369.101430] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2370.642806] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 2372.913839] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2374.629603] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2377.367475] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2377.523156] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2379.559033] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:161) [ 2379.783048] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 2382.024273] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2389.799829] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:170 to 0x280000401:193) [ 2407.231784] Lustre: lustre-MDT0001: evicting 279c08db-98cb-43b9-9970-e9d948e956c4 (at 192.168.202.47@tcp) by administrative request [ 2407.237211] Lustre: Skipped 1 previous similar message [ 2436.066774] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2436.067378] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2436.067980] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2436.072256] Lustre: Skipped 1 previous similar message [ 2436.075158] LustreError: Skipped 1 previous similar message [ 2437.742225] LustreError: 146955:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 2437.797330] Lustre: server umount lustre-OST0000 complete [ 2440.673534] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2440.676539] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2440.681794] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2440.684155] Lustre: Skipped 1 previous similar message [ 2445.500076] LustreError: 147158:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup [ 2445.502343] LustreError: 147158:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 2445.533382] Lustre: server umount lustre-MDT0000 complete [ 2446.303841] LustreError: 144869:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2446.308870] LustreError: 144869:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 2446.916215] LustreError: 144132:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1762168828 with bad export cookie 16435762485134589270 [ 2446.917445] LustreError: MGC192.168.202.147@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2446.921769] LustreError: 144132:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 2446.990353] LustreError: 147360:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [ 2446.992829] LustreError: 147360:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 2447.068644] Lustre: server umount lustre-MDT0001 complete [ 2451.273122] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing unload_modules_local [ 2452.448557] Key type lgssc unregistered [ 2452.581413] LNet: 147891:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2452.584632] LNetError: 147891:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2452.594515] LNet: Removed LNI 192.168.202.147@tcp [ 2452.915293] Key type .llcrypt unregistered [ 2452.916665] Key type ._llcrypt unregistered [ 2482.874843] Key type ._llcrypt registered [ 2482.876339] Key type .llcrypt registered [ 2482.927481] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing load_modules_local [ 2483.304859] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2483.368575] alg: No test for adler32 (adler32-zlib) [ 2484.240120] Lustre: Lustre: Build Version: 2.16.59_50_g4204c43 [ 2484.337236] LNet: Added LNI 192.168.202.147@tcp [8/256/0/180] [ 2485.927192] Key type lgssc registered [ 2486.401911] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2489.542853] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 2489.553173] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2490.763204] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2492.210938] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 2495.435802] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2495.586572] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2497.003253] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 2499.038299] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2500.463571] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2503.185732] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2503.357062] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2505.384798] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:193) [ 2505.657341] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 2507.686302] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2516.645374] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:195 to 0x280000401:225) [ 2521.056232] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2521.059699] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2521.064992] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2522.085650] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2522.085789] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2522.085978] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2522.096032] Lustre: Skipped 1 previous similar message [ 2522.927322] LustreError: 152608:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 2522.962960] Lustre: server umount lustre-OST0000 complete [ 2526.176121] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2526.178430] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2526.182845] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2526.397136] LustreError: 152810:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup [ 2526.400077] LustreError: 152810:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 2526.427729] Lustre: server umount lustre-MDT0000 complete [ 2527.199939] LustreError: 150333:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2527.205756] LustreError: 150333:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 2527.671084] LustreError: 150314:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1762168909 with bad export cookie 12270458862678715293 [ 2527.675177] LustreError: MGC192.168.202.147@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2527.675687] LustreError: 150314:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2527.740854] LustreError: 153013:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [ 2527.743391] LustreError: 153013:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 2527.793426] Lustre: server umount lustre-MDT0001 complete [ 2531.368976] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing unload_modules_local [ 2532.363326] Key type lgssc unregistered [ 2532.490432] LNet: 153543:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2532.493100] LNetError: 153543:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2532.499384] LNet: Removed LNI 192.168.202.147@tcp [ 2532.817882] Key type .llcrypt unregistered [ 2532.818895] Key type ._llcrypt unregistered [ 2541.887855] Key type ._llcrypt registered [ 2541.888904] Key type .llcrypt registered [ 2541.933304] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing load_modules_local [ 2542.272078] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2542.289085] alg: No test for adler32 (adler32-zlib) [ 2543.147614] Lustre: Lustre: Build Version: 2.16.59_50_g4204c43 [ 2543.232928] LNet: Added LNI 192.168.202.147@tcp [8/256/0/180] [ 2544.823174] Key type lgssc registered [ 2545.222316] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2547.763545] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2550.412770] Lustre: DEBUG MARKER: == conf-sanity test 100: check lshowmount lists MGS, MDT, OST and 0@lo ========================================================== 06:22:11 (1762168931) [ 2556.172988] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_hostid [ 2559.750727] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing load_modules_local [ 2563.362516] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2565.696609] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2567.948713] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2570.210908] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2572.689075] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2572.717028] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 2572.723134] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2573.809280] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2573.823092] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2573.864178] Lustre: lustre-MDT0000: new disk, initializing [ 2573.898617] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2573.913432] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2575.458079] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 2580.100368] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2580.126694] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2580.163866] Lustre: 157750:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2580.191893] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 2580.194760] Lustre: Skipped 1 previous similar message [ 2580.235128] Lustre: lustre-MDT0001: new disk, initializing [ 2580.274247] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2580.287238] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 2580.291278] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2581.798327] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 2585.126290] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2586.372828] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2588.802213] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 2588.830700] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2588.934609] Lustre: lustre-OST0000: new disk, initializing [ 2588.937119] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 2588.970615] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2590.335189] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 2590.339080] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2590.380491] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2591.067868] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 2594.351545] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2600.929578] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2600.931661] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2600.932657] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2600.934857] Lustre: Skipped 1 previous similar message [ 2604.911127] LustreError: 160338:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 2604.957442] Lustre: server umount lustre-OST0000 complete [ 2611.169696] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2611.169790] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2611.170448] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2611.170454] Lustre: Skipped 1 previous similar message [ 2611.171890] LustreError: Skipped 1 previous similar message [ 2611.184810] Lustre: Skipped 1 previous similar message [ 2612.420257] LustreError: 160539:0:(obd_class.h:479:obd_check_dev()) Device 22 not setup [ 2612.422917] LustreError: 160539:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 2612.461569] Lustre: server umount lustre-MDT0000 complete [ 2613.856241] LustreError: 157741:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1762168995 with bad export cookie 1951124710477250891 [ 2613.859316] LustreError: MGC192.168.202.147@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2613.860263] LustreError: 157741:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 2613.924757] LustreError: 160741:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 2613.926620] LustreError: 160741:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 2613.983180] Lustre: server umount lustre-MDT0001 complete [ 2617.802269] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing unload_modules_local [ 2618.890197] Key type lgssc unregistered [ 2619.030365] LNet: 161271:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2619.033857] LNetError: 161271:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2619.043424] LNet: Removed LNI 192.168.202.147@tcp [ 2619.368593] Key type .llcrypt unregistered [ 2619.369846] Key type ._llcrypt unregistered [ 2629.212941] Key type ._llcrypt registered [ 2629.214464] Key type .llcrypt registered [ 2629.262210] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing load_modules_local [ 2629.570421] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2629.578646] alg: No test for adler32 (adler32-zlib) [ 2630.446575] Lustre: Lustre: Build Version: 2.16.59_50_g4204c43 [ 2630.537705] LNet: Added LNI 192.168.202.147@tcp [8/256/0/180] [ 2632.119292] Key type lgssc registered [ 2632.547609] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2635.567848] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 2635.578405] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2636.763156] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2638.212996] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 2641.320072] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2641.449121] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2642.887079] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 2644.928602] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2646.254849] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2648.643088] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 2648.761960] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2650.813445] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 2652.809536] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2662.057402] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:3 to 0x280000401:33) [ 2666.005604] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 2666.013664] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2666.020852] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 2666.024405] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2666.029718] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 2666.437176] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2666.810995] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 2666.814917] Lustre: Skipped 1 previous similar message [ 2666.817114] LustreError: 162663:0:(osp_precreate.c:651:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 2666.820917] LustreError: 162663:0:(osp_precreate.c:1355:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 2666.822949] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2666.830038] Lustre: Skipped 1 previous similar message [ 2666.832066] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 2666.834990] Lustre: Skipped 1 previous similar message [ 2666.837985] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2666.843466] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 2666.846206] Lustre: Skipped 1 previous similar message [ 2667.201604] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2667.552501] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2667.930395] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 2667.933765] Lustre: Skipped 2 previous similar messages [ 2667.941685] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2667.947055] Lustre: Skipped 2 previous similar messages [ 2667.949092] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 2667.952231] Lustre: Skipped 2 previous similar messages [ 2667.954155] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2667.959188] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 2667.961414] Lustre: Skipped 2 previous similar messages [ 2668.331139] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2668.713328] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2669.121973] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2669.495509] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2669.881452] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2670.224259] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 2670.227315] Lustre: Skipped 5 previous similar messages [ 2670.233915] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2670.238215] Lustre: Skipped 5 previous similar messages [ 2670.240168] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 2670.242508] Lustre: Skipped 5 previous similar messages [ 2670.243975] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2670.247600] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 2670.249198] Lustre: Skipped 5 previous similar messages [ 2670.587956] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2670.952862] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2671.309198] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2671.646691] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2671.989856] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2672.345318] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2672.688524] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2673.037281] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2673.425332] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2673.770767] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2674.125317] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2674.475826] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 2674.480254] Lustre: Skipped 11 previous similar messages [ 2674.488232] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2674.494213] Lustre: Skipped 11 previous similar messages [ 2674.496969] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 2674.500381] Lustre: Skipped 11 previous similar messages [ 2674.502752] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2674.508179] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 2674.511206] Lustre: Skipped 11 previous similar messages [ 2674.888938] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2675.250905] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2675.610883] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2675.962651] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2676.311664] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2676.674413] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2677.071065] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2677.436511] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2677.817292] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2678.231590] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2678.587749] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2678.958937] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2679.312747] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2679.667757] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2680.029610] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2680.383224] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2680.748148] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2681.123328] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2681.473606] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2681.830483] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2682.184555] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2682.514143] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 2682.516634] Lustre: Skipped 21 previous similar messages [ 2682.523624] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2682.528186] Lustre: Skipped 21 previous similar messages [ 2682.530230] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 2682.533066] Lustre: Skipped 21 previous similar messages [ 2682.534424] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2682.537882] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 2682.539544] Lustre: Skipped 21 previous similar messages [ 2682.883870] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2683.269672] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2683.639509] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2684.023843] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2684.396548] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2684.765819] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2685.128734] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2685.481343] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2685.834215] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2686.180584] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2686.534104] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2686.889857] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2687.269230] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2687.605880] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2687.932919] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2688.291965] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2688.647441] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2688.994420] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2689.332859] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2689.668459] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2690.011432] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2690.380989] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2690.742450] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2691.082326] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2691.431469] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2691.774152] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2692.127351] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2692.474219] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2692.824042] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2693.192755] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2693.573337] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2693.921699] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2694.284485] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2694.642908] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2695.007943] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2695.347254] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2695.692736] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2696.063439] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2696.438308] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2696.780417] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2697.127411] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2697.487772] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2697.830080] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2698.203919] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2698.564733] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 2698.567542] Lustre: Skipped 44 previous similar messages [ 2698.573992] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2698.578687] Lustre: Skipped 44 previous similar messages [ 2698.580961] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 2698.584116] Lustre: Skipped 44 previous similar messages [ 2698.585793] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2698.591098] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 2698.593995] Lustre: Skipped 44 previous similar messages [ 2698.944409] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2699.310410] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2699.700960] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2700.053824] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2700.392318] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2700.782149] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2701.204639] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2701.592308] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2702.028478] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2702.534914] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2702.956309] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2703.385550] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2703.820148] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2704.238214] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2704.653296] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2705.070218] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2705.485686] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2705.888024] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2706.271288] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2706.692573] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2707.104414] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2707.516555] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2707.922395] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2708.364130] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2708.796287] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2709.234344] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2709.674573] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2710.088989] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2710.509221] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2710.974115] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2711.356711] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2711.716778] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2712.077560] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2712.467420] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2712.864246] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2713.275240] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2713.665325] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2714.059718] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2714.431632] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2714.832128] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2715.226091] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2715.597784] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2715.974178] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2716.322244] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2716.662047] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2717.006241] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2717.368274] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2717.722824] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2718.073872] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2718.440640] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2718.790744] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2719.153360] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2719.536179] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2719.925409] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2720.293317] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2720.667909] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2721.124648] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2721.561659] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2721.976036] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2722.476169] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2722.877238] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2723.302246] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2723.767532] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2724.291627] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2724.786701] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2725.279217] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2725.775284] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2726.308599] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2727.035458] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2727.902226] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2728.713146] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2729.522785] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2730.299823] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2731.140627] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 2731.148361] Lustre: Skipped 73 previous similar messages [ 2731.160955] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2731.168615] Lustre: Skipped 73 previous similar messages [ 2731.176163] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 2731.183350] Lustre: Skipped 73 previous similar messages [ 2731.192161] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2731.201298] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 2731.211395] Lustre: Skipped 73 previous similar messages [ 2732.066243] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2732.870825] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2733.911045] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2734.871113] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2736.119159] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2737.365841] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2738.362346] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2739.544129] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2740.719225] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2741.926434] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2742.983049] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2744.030068] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2744.997088] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2746.302219] LustreError: 162663:0:(osp_precreate.c:651:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 2746.319867] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2746.329360] LustreError: 162663:0:(osp_precreate.c:1355:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 2747.434428] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2748.396641] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2749.281662] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2750.159533] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2751.175587] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2752.250322] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2753.080774] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2754.330840] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2755.332197] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2756.078907] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2756.783964] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2757.784916] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2758.646532] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2759.670453] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2760.483106] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2761.399874] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2762.388388] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2763.563739] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2764.543890] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2765.401269] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2766.373981] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2767.514843] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2768.383251] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2769.293180] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2770.258914] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2771.331264] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2772.268669] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2773.204809] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2773.998768] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2774.852054] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2775.727611] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2776.683945] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2777.496921] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2778.492723] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2779.482947] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2780.425609] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2781.285803] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2781.853682] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2782.595232] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2783.332052] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2783.947100] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2784.581505] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2785.208403] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2785.767303] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2786.315729] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2786.874967] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2787.412029] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2787.954589] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2788.535544] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2789.076565] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2789.559959] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2790.078228] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2790.627153] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2791.234760] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2791.851766] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2792.465682] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2793.043991] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2793.633993] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2794.175464] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2794.705972] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2795.244035] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 2795.252514] Lustre: Skipped 74 previous similar messages [ 2795.263306] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2795.269491] Lustre: Skipped 74 previous similar messages [ 2795.275692] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 2795.278529] Lustre: Skipped 74 previous similar messages [ 2795.281718] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2795.289599] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 2795.292445] Lustre: Skipped 74 previous similar messages [ 2795.825054] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2796.365118] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2796.918288] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2797.426468] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2797.910414] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2798.416445] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2798.898436] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2799.453907] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2800.045365] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2800.672541] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2801.283389] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2801.904854] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2802.424182] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2802.949706] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2803.453764] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2803.964948] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2804.496234] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2805.020236] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2805.629069] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2806.203573] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2806.739480] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2807.299549] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2807.815718] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2808.335566] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2808.845529] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2809.305751] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2809.955445] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2810.512504] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2811.060426] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2811.628576] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2812.227820] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2812.736509] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2813.252531] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2813.810852] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2814.332591] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2814.935270] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2815.452829] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2816.021371] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2816.642623] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2817.214768] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2817.761625] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2818.255529] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2818.837117] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2819.334059] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2819.782404] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2820.338791] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2820.873745] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2821.361874] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2821.894067] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2822.387823] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2822.899943] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2823.431162] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2823.931357] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2824.436640] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2825.020383] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2825.527442] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2826.016567] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2826.584837] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2827.260283] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2827.964055] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2828.600683] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2829.155063] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2829.692740] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2830.294357] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2830.851093] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2831.456718] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2832.021262] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2832.659955] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2833.186842] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2833.753340] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2834.323511] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2834.874235] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2835.394351] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2835.968105] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2836.508966] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2837.046567] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2837.553859] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2838.062132] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2838.547587] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2839.111918] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2839.681547] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2840.195944] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2840.738404] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2841.315763] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2841.889487] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2842.480143] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2843.155525] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2843.753601] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2844.361168] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2844.990916] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2845.539586] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2846.054261] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2846.625254] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2847.166323] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2847.700274] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2848.259396] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2848.830862] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2849.354658] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2849.933378] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2850.497913] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2851.129066] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2851.681422] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2852.236125] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2852.898229] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2853.510582] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2854.079320] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2854.624879] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2855.145456] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2855.716543] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2856.202324] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2856.766693] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2857.415311] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2858.089292] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2858.739692] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2859.359096] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2859.891182] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2860.422618] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2860.918753] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2861.467082] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2861.994335] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2862.591318] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2863.155724] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2863.741452] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2864.302065] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2864.776234] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2865.272341] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2865.802395] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2866.318071] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2866.803469] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2867.333156] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2867.819984] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2868.400436] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2869.022432] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2869.662750] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2870.181901] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2870.707886] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2871.258477] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2871.781928] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2872.295410] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2872.787662] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2873.361740] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2873.926900] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2874.437161] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2874.926920] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2875.435877] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2875.944847] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2876.374213] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2876.790269] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2877.291293] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2877.836373] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2878.395309] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2878.948797] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2879.430434] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2879.911696] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2880.370978] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2880.893833] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2881.419437] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2881.918970] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2882.436546] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2882.937256] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2883.503187] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2884.041800] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2884.577484] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2885.014586] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2885.543530] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2886.049293] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2886.523779] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2887.071657] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2887.526583] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2887.963260] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2888.426804] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2888.884745] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2889.357109] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2889.795465] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2890.217332] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2890.666181] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2891.081343] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2891.465324] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2891.840417] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2892.225854] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2892.597885] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2892.953305] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2893.325779] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2893.688499] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2894.049063] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2894.437088] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2894.805642] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2895.176847] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2895.643896] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2896.057779] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2896.494914] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2896.958859] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2897.399453] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2897.922127] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2898.432642] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2898.955117] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2899.472488] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2899.974594] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2900.428642] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2900.926372] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2901.417867] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2901.948776] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2902.509124] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2903.052314] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2903.661910] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2904.234758] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2904.819449] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2905.562389] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2906.213265] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2906.781985] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2907.448331] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2908.069634] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2908.672874] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2909.356133] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2909.987583] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2910.520277] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2911.075778] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2911.587097] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2912.095212] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2912.696643] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2913.250309] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2913.780110] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2914.339322] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2914.806222] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2915.308407] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2915.902976] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2916.451094] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2917.060913] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2917.688554] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2918.191534] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2918.811396] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2919.365033] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2919.912221] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2920.401578] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2920.897223] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2921.351161] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2921.773548] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2922.230183] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2922.695484] LustreError: 162663:0:(osp_precreate.c:651:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 2922.705082] LustreError: 162663:0:(osp_precreate.c:1355:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 2922.706956] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2923.173591] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2923.668724] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 2923.671513] Lustre: Skipped 240 previous similar messages [ 2923.681116] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2923.686113] Lustre: Skipped 240 previous similar messages [ 2923.689016] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 2923.692938] Lustre: Skipped 240 previous similar messages [ 2923.695378] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2923.699773] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 2923.702246] Lustre: Skipped 240 previous similar messages [ 2924.191270] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2924.663836] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2925.135815] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2925.683200] LustreError: 162663:0:(osp_precreate.c:651:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 2925.688644] LustreError: 162663:0:(osp_precreate.c:1355:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 2925.699049] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2926.242658] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2926.818462] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2927.352404] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2927.868823] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2928.351287] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2928.868758] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2929.337280] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2929.831640] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2930.280380] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2930.746416] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2931.189365] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2931.655160] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2932.091727] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2932.528589] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2932.953367] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2933.423462] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2933.842279] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2934.297656] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2934.747824] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2935.243463] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2935.727648] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2936.224470] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2936.677753] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2937.151323] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2937.685147] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2938.175463] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2938.721896] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2939.274537] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2939.858431] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2940.427713] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2940.975991] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2941.466481] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2941.977273] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2942.500908] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2943.019151] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2943.568892] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2944.124859] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2944.650201] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2945.113193] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2945.580737] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2946.073829] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2946.593779] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2947.236605] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2947.810317] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2948.366475] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2948.977566] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2949.530216] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2950.094990] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2950.614124] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2951.128068] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2951.589687] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2952.100924] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2952.593758] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2953.132122] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2953.719712] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2954.168673] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2954.629721] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2955.085444] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2955.519293] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2955.999521] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2956.493503] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2956.943356] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2957.385642] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2957.878605] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2958.335281] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2958.780933] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2959.226853] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2959.709240] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2960.159304] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2960.641837] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2961.095711] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2961.628687] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2962.094254] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2962.589052] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2963.042585] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2963.492638] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2963.948404] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2964.464984] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2964.901120] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2965.356649] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2965.814284] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2966.284176] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2966.710569] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2967.164957] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2967.595116] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2968.080827] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2968.540325] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2969.079771] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2969.611460] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2970.174889] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2970.721368] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2971.200298] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2971.724142] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2972.178960] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2972.694023] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2973.259574] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2973.750067] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2974.232259] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2974.762489] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2975.197449] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2975.657742] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2976.142989] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2976.605677] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2977.085239] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2977.540196] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2978.022028] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2978.561599] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2979.024100] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2979.446721] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2979.903697] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2980.409383] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2980.887894] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2981.342083] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2981.806428] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2982.299508] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2982.780706] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2983.247261] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2983.727881] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2984.166500] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2984.594825] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2985.078858] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2985.534491] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2985.979681] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2986.472369] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2986.943128] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2987.437196] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2987.899724] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2988.322045] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2988.728785] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2989.199971] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2989.668468] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2990.137920] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2990.592309] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2991.086594] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2991.626210] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2992.153837] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2992.652599] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2993.134420] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2993.607043] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2994.056916] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2994.598095] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2995.084344] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2995.585331] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2996.141864] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2996.758903] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2997.354591] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2997.911646] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2998.452717] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2998.954266] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2999.485489] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3000.043373] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3000.571732] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3001.080061] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3001.644656] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3002.172903] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3002.724056] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3003.248799] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3003.826540] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3004.428681] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3004.937081] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3005.420652] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3005.962558] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3006.412081] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3006.885893] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3007.368901] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3007.874285] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3008.445732] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3009.008509] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3009.580426] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3010.160828] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3010.657453] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3011.207705] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3011.753368] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3012.351037] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3012.936127] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3013.547948] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3014.087736] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3014.714893] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3015.299802] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3015.792148] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3016.297672] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3016.796639] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3017.293614] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3017.815735] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3018.335926] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3018.891743] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3019.465302] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3019.922555] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3020.344348] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3020.804315] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3021.311575] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3021.827241] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3022.371377] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3022.827915] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3023.290107] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3023.784247] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3024.336912] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3024.778165] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3025.248894] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3025.726133] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3026.215170] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3026.687077] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3027.226899] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3027.775909] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3028.329567] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3028.803234] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3029.282945] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3029.898351] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3030.421171] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3030.957729] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3031.548954] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3032.188807] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3032.733433] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3033.277333] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3033.847987] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3034.357598] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3034.895438] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3035.450934] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3035.949373] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3036.448321] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3036.910828] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3037.377259] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3037.864188] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3038.329329] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3038.805109] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3039.277433] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3039.824447] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3040.334162] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3040.775933] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3041.216122] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3041.663402] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3042.114840] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3042.504219] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3042.918464] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3043.332061] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3043.790159] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3044.353611] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3044.899228] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3045.409982] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3045.953512] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3046.485757] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3046.999926] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3047.515147] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3048.012324] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3048.505378] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3049.019025] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3049.449190] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3049.883784] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3050.327764] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3050.788563] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3051.240038] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3051.786753] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3052.224372] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3052.681054] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3053.136743] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3053.595902] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3054.033506] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3054.461559] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3054.872389] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3055.302136] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3055.699752] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3056.129632] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3056.585654] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3057.023277] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3057.454052] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3057.907219] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3058.388475] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3058.848639] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3059.251291] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3059.715147] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3060.172138] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3060.631355] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3061.075456] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3061.506684] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3061.942275] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3062.364045] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3062.751459] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3063.177350] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3063.608830] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3064.078564] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3064.585706] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3065.003665] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3065.480914] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3066.000193] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3066.510098] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3066.969818] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3067.414075] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3067.904763] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3068.359407] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3068.850586] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3069.309231] LustreError: 162663:0:(osp_precreate.c:651:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 3069.315789] LustreError: 162663:0:(osp_precreate.c:1355:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 3069.316846] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3069.766962] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3070.223389] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3070.682499] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3071.210921] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3071.740176] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3072.313103] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3072.774840] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3073.280683] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3073.759610] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3074.292144] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3074.749933] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3075.189171] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3075.644980] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3076.108924] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3076.602644] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3077.080149] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3077.538740] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3077.938575] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3078.328891] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3078.745164] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3079.147955] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3079.570627] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3079.992879] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3080.433197] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3080.893801] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3081.342438] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3081.796911] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3082.234355] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3082.691970] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3083.107116] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3083.507269] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3083.899830] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3084.301189] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3084.671235] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3085.044102] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3085.427915] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3085.839867] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3086.273394] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3086.694880] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3087.081413] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3087.482156] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3087.878220] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3088.279517] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3088.713109] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3089.183381] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3089.614348] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3090.069735] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3090.513403] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3090.965401] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3091.390235] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3091.850651] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3092.291511] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3092.698852] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3093.089455] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3093.487567] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3093.874410] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3094.253342] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3094.656249] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3095.067491] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3095.465528] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3095.899606] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3096.342254] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3096.738222] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3097.142446] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3097.593242] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3098.037612] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3098.501078] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3099.009217] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3099.428303] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3099.852311] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3100.275028] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3100.740276] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3101.228595] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3101.700892] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3102.155199] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3102.600529] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3103.101171] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3103.581804] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3104.019570] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3104.475863] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3104.939549] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3105.342138] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3105.764430] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3106.153266] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3106.575288] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3107.000971] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3107.427200] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3107.877181] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3108.294200] LustreError: 162663:0:(osp_precreate.c:651:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 3108.299481] LustreError: 162663:0:(osp_precreate.c:1355:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 3108.299709] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3108.727219] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3109.153320] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3109.562859] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3109.943317] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3110.348303] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3110.742218] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3111.130204] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3111.551416] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3111.939996] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3112.326803] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3112.716285] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3113.115842] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3113.505413] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3113.910354] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3114.312544] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3114.732726] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3115.175899] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3115.587562] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3116.070259] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3116.551800] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3117.083688] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3117.539412] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3117.967599] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3118.419291] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3118.845378] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3119.311959] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3119.735972] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3120.175761] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3120.619340] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3121.069098] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3121.480367] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3121.883530] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3122.293433] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3122.700241] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3123.089296] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3123.479613] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3123.871731] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3124.230428] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3124.599658] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3124.960378] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3125.336671] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3125.708738] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3126.087959] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3126.515301] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3126.917774] LustreError: 162663:0:(osp_precreate.c:651:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 3126.922645] LustreError: 162663:0:(osp_precreate.c:1355:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 3126.923170] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3127.366706] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3127.764709] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3128.142457] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3128.549656] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3128.944084] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3129.351447] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3129.734502] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3130.139639] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3130.547137] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3130.916752] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3131.351236] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3131.777348] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3132.201855] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3132.633458] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3133.055298] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3133.432748] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3133.816382] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3134.197437] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3134.598279] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3135.010792] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3135.460529] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3135.850484] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3136.337276] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3136.770219] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3137.188579] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3137.614218] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3138.038728] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3138.432689] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3138.825356] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3139.260664] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3139.675351] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3140.078587] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3140.481341] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3140.878342] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3141.252353] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3141.653705] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3142.066857] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3142.454952] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3142.854549] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3143.253910] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3143.695107] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3144.127685] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3144.524349] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3144.909252] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3145.293111] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3145.656109] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3146.008132] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3146.400234] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3146.805450] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3147.190755] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3147.575346] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3147.961512] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3148.354291] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3148.732891] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3149.110636] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3149.510505] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3149.907777] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3150.308437] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3150.746461] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3151.162736] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3151.635604] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3152.036422] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3152.479638] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3152.947218] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3153.373564] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3153.807255] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3154.237704] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3154.639506] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3155.042330] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3155.452399] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3155.893985] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3156.298835] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3156.743708] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3157.199622] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3157.645471] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3158.052240] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3158.473041] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3158.897163] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3159.323508] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3159.733746] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3160.178554] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3160.609589] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3161.067888] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3161.574054] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3162.042098] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3162.499587] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3162.970830] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3163.438088] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3163.866621] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3164.305429] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3463.886395] LustreError: 164584:0:(ofd_dev.c:1780:ofd_create_hdl()) lustre-OST0000: unable to precreate: rc = -28 [ 3463.892463] LustreError: 162663:0:(osp_precreate.c:651:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -28 [ 3639.263910] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3639.267834] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3639.272754] Lustre: Skipped 519 previous similar messages [ 3639.275224] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3641.272812] LustreError: 216473:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 3641.313608] LustreError: 216287:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3641.319558] LustreError: 216287:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 3641.354137] Lustre: server umount lustre-OST0000 complete [ 3645.407926] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3645.411919] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3646.432107] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3646.434969] Lustre: Skipped 1 previous similar message [ 3651.552395] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3651.555016] Lustre: Skipped 1 previous similar message [ 3656.674946] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3656.678447] Lustre: Skipped 1 previous similar message [ 3657.183147] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 3657.236897] LustreError: 216673:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup [ 3657.239394] LustreError: 216673:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 3657.344508] Lustre: server umount lustre-MDT0000 complete [ 3659.099429] LustreError: 164263:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1762170040 with bad export cookie 11977408394429545190 [ 3659.101675] LustreError: MGC192.168.202.147@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3659.104274] LustreError: 164263:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 3659.177067] LustreError: 216875:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [ 3659.180128] LustreError: 216875:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 3659.243821] Lustre: server umount lustre-MDT0001 complete [ 3663.364044] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing unload_modules_local [ 3664.590493] Key type lgssc unregistered [ 3664.745876] LNet: 217406:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3664.749849] LNetError: 217406:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3664.762644] LNet: Removed LNI 192.168.202.147@tcp [ 3665.121505] Key type .llcrypt unregistered [ 3665.123116] Key type ._llcrypt unregistered [ 3676.160523] Key type ._llcrypt registered [ 3676.162063] Key type .llcrypt registered [ 3676.220686] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing load_modules_local [ 3676.603258] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3676.612503] alg: No test for adler32 (adler32-zlib) [ 3677.489803] Lustre: Lustre: Build Version: 2.16.59_50_g4204c43 [ 3677.596237] LNet: Added LNI 192.168.202.147@tcp [8/256/0/180] [ 3679.183243] Key type lgssc registered [ 3679.643128] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3682.794151] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3682.802598] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3683.997923] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3685.552157] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 3689.203031] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3689.372076] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3691.001907] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 3693.298784] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 3694.848544] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 3697.770445] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3697.922507] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3700.176776] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 3702.633389] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3709.228750] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49666 to 0x280000401:49697) [ 3716.204476] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 3716.919226] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3716.925233] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 3716.929099] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3716.934549] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 3716.934591] LustreError: 218801:0:(osp_precreate.c:1263:osp_precreate_thread()) cfs_fail_timeout id 2107 sleeping for 20000ms [ 3719.255717] Lustre: lustre-OST0000: Not available for connect from 192.168.202.47@tcp (stopping) [ 3719.649099] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3719.650523] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3723.765089] LustreError: 221200:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 3723.815347] Lustre: server umount lustre-OST0000 complete [ 3737.015735] LustreError: 218801:0:(osp_precreate.c:1263:osp_precreate_thread()) cfs_fail_timeout id 2107 awake [ 3752.699272] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3752.857218] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3755.241969] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 3757.483151] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3760.037244] LustreError: lustre-OST0000-osc-MDT0001: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3760.043466] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3760.043579] Lustre: lustre-OST0000-osc-MDT0001: Connection restored to 0@lo (at 0@lo) [ 3760.050063] Lustre: Skipped 1 previous similar message [ 3760.214110] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid in FULL state after 2 sec [ 3761.843693] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff94f444f34000.ost_server_uuid 50 [ 3762.544096] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff94f444f34000.ost_server_uuid in FULL state after 0 sec [ 3765.218633] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3765.219536] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3765.223599] Lustre: Skipped 2 previous similar messages [ 3765.225820] Lustre: Skipped 2 previous similar messages [ 3765.229576] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3769.966180] LustreError: 222517:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 3769.968198] LustreError: 222517:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 3770.012081] Lustre: server umount lustre-OST0000 complete [ 3775.457369] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3775.458413] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3775.462414] Lustre: Skipped 1 previous similar message [ 3775.465203] Lustre: Skipped 1 previous similar message [ 3777.667331] LustreError: 222718:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup [ 3777.670132] LustreError: 222718:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 3777.702446] Lustre: server umount lustre-MDT0000 complete [ 3779.150623] LustreError: 220397:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1762170160 with bad export cookie 15955323511273879232 [ 3779.153304] LustreError: MGC192.168.202.147@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3779.155826] LustreError: 220397:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 3779.277509] Lustre: server umount lustre-MDT0001 complete [ 3783.498976] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing unload_modules_local [ 3784.772963] Key type lgssc unregistered [ 3784.925190] LNet: 223452:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3784.929942] LNetError: 223452:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3784.939133] LNet: Removed LNI 192.168.202.147@tcp [ 3785.334815] Key type .llcrypt unregistered [ 3785.336548] Key type ._llcrypt unregistered [ 3799.321260] Key type ._llcrypt registered [ 3799.322712] Key type .llcrypt registered [ 3799.376103] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing load_modules_local [ 3799.864415] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3799.875216] alg: No test for adler32 (adler32-zlib) [ 3800.762392] Lustre: Lustre: Build Version: 2.16.59_50_g4204c43 [ 3800.866312] LNet: Added LNI 192.168.202.147@tcp [8/256/0/180] [ 3802.463211] Key type lgssc registered [ 3802.913361] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3806.617151] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3806.629311] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3807.846369] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3809.387954] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 3812.887880] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3813.113116] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3814.780840] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 3816.054196] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 3819.082598] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3819.269382] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3821.750922] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 3825.507284] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3825.532958] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3825.607136] Lustre: lustre-OST0001: new disk, initializing [ 3825.609449] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 3825.647909] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 3828.069745] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 3828.890932] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 3828.895160] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 3828.943780] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 3833.004849] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49699 to 0x280000401:49729) [ 3834.431341] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 3838.663061] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 3853.794090] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3853.796875] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3853.799322] Lustre: Skipped 3 previous similar messages [ 3853.801756] Lustre: Skipped 2 previous similar messages [ 3854.025560] LustreError: 228504:0:(obd_class.h:479:obd_check_dev()) Device 28 not setup [ 3854.064968] Lustre: server umount lustre-MDT0000 complete [ 3855.667581] LustreError: 225120:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1762170237 with bad export cookie 8117473437442506263 [ 3855.669915] LustreError: MGC192.168.202.147@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3855.671761] LustreError: 225120:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 3855.917033] LustreError: 228705:0:(obd_class.h:479:obd_check_dev()) Device 27 not setup [ 3855.919644] LustreError: 228705:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 3855.987102] Lustre: server umount lustre-MDT0001 complete [ 3867.127987] LustreError: 228905:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 3867.131021] LustreError: 228905:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 3867.161988] Lustre: server umount lustre-OST0000 complete [ 3879.028925] LustreError: 229107:0:(obd_class.h:479:obd_check_dev()) Device 2 not setup [ 3879.032040] LustreError: 229107:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 3879.096324] Lustre: server umount lustre-OST0001 complete [ 3880.549228] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro [ 3880.588068] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3881.094927] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro [ 3881.132160] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3881.614361] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: ,errors=remount-ro [ 3881.660245] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3882.106989] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: ,errors=remount-ro [ 3882.141113] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3887.699460] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing load_modules_local [ 3891.943277] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3892.140648] LustreError: 230514:0:(ldlm_lib.c:1179:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3892.150286] LustreError: 230514:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 3892.174267] Lustre: mylustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3893.695775] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 3897.015531] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3898.684205] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 3899.834444] Lustre: 231623:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 3902.523054] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3902.669814] Lustre: mylustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3902.672987] Lustre: Skipped 1 previous similar message [ 3905.021118] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 3906.791633] LustreError: 232533:0:(ldlm_lib.c:1179:target_handle_connect()) mylustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3906.796903] Lustre: mylustre-OST0000: new connection from mylustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:100 to 0x280000400:129) [ 3908.685759] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3911.251696] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 3916.845984] Lustre: mylustre-OST0000: new connection from mylustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49699 to 0x280000401:49761) [ 3920.167171] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 3921.690851] Lustre: 233469:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 3932.131689] Lustre: mylustre-MDT0000-osp-MDT0001: Connection to mylustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3932.133857] Lustre: mylustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3932.136834] Lustre: Skipped 3 previous similar messages [ 3932.141404] Lustre: Skipped 3 previous similar messages [ 3937.248360] Lustre: mylustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3937.251048] Lustre: Skipped 2 previous similar messages [ 3938.115825] LustreError: 233778:0:(obd_class.h:479:obd_check_dev()) Device 10 not setup [ 3938.117908] LustreError: 233778:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 3938.151623] Lustre: server umount mylustre-MDT0000 complete [ 3939.652408] LustreError: 230489:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1762170321 with bad export cookie 8117473437442509112 [ 3939.656280] LustreError: MGC192.168.202.147@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3939.657354] LustreError: 230489:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 3939.802826] Lustre: server umount mylustre-MDT0001 complete [ 3951.548092] LustreError: 234179:0:(obd_class.h:479:obd_check_dev()) Device 23 not setup [ 3951.550695] LustreError: 234179:0:(obd_class.h:479:obd_check_dev()) Skipped 15 previous similar messages [ 3951.578888] Lustre: server umount mylustre-OST0000 complete [ 3963.435967] Lustre: server umount mylustre-OST0001 complete [ 3964.624659] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro [ 3964.657057] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3965.085335] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro [ 3965.123793] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3965.564042] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: ,errors=remount-ro [ 3965.600760] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3966.058693] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: ,errors=remount-ro [ 3966.095897] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3971.880416] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing load_modules_local [ 3975.761331] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3975.987701] LustreError: 235790:0:(ldlm_lib.c:1179:target_handle_connect()) tfs-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3976.028601] Lustre: tfs-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3976.031730] Lustre: Skipped 1 previous similar message [ 3977.631301] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 3981.170078] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3981.280824] Lustre: tfs-MDT0001: Not available for connect from 0@lo (not set up) [ 3982.968335] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 3984.221289] Lustre: 236901:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 3986.808465] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3988.003407] LustreError: 237259:0:(ldlm_lib.c:1179:target_handle_connect()) tfs-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3988.009700] LustreError: 237259:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 3988.014402] Lustre: tfs-OST0000: new connection from tfs-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49699 to 0x280000401:49793) [ 3989.186130] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 3991.078084] Lustre: tfs-OST0000: new connection from tfs-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:131 to 0x280000400:161) [ 3992.284244] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 3994.663513] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 3998.124145] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 3999.651357] Lustre: 238748:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 4006.879937] LustreError: tfs-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4006.882662] Lustre: tfs-MDT0000-osp-MDT0001: Connection to tfs-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4006.888377] Lustre: tfs-MDT0000: Not available for connect from 0@lo (stopping) [ 4013.060058] LustreError: 238996:0:(obd_class.h:479:obd_check_dev()) Device 10 not setup [ 4013.062268] LustreError: 238996:0:(obd_class.h:479:obd_check_dev()) Skipped 9 previous similar messages [ 4013.104292] Lustre: server umount tfs-MDT0000 complete [ 4014.499500] LustreError: 235764:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1762170395 with bad export cookie 8117473437442511065 [ 4014.503317] LustreError: MGC192.168.202.147@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4014.503748] LustreError: 235764:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4014.630217] Lustre: server umount tfs-MDT0001 complete [ 4026.376028] Lustre: server umount tfs-OST0000 complete [ 4038.113112] Lustre: server umount tfs-OST0001 complete [ 4039.348270] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro [ 4039.380058] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4039.816777] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro [ 4039.855493] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4040.299606] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: ,errors=remount-ro [ 4040.330710] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4040.759302] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: ,errors=remount-ro [ 4040.790687] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4046.081663] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing load_modules_local [ 4049.961953] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4050.150149] LustreError: 241008:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4050.158575] LustreError: 241008:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 4050.178511] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4050.180846] Lustre: Skipped 3 previous similar messages [ 4051.603456] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 4054.594187] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4056.103844] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 4057.141438] Lustre: 242117:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 4059.527894] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4060.708481] LustreError: 242471:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4060.716084] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49699 to 0x280000401:49825) [ 4061.698343] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 4064.681957] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4065.834839] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:131 to 0x280000400:193) [ 4066.849869] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 4070.367935] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 4071.624142] Lustre: 243964:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 4079.122383] Lustre: DEBUG MARKER: == conf-sanity test 104a: Make sure user defined options are reflected in mount ========================================================== 06:47:40 (1762170460) [ 4094.436203] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4094.438335] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4094.440455] Lustre: Skipped 6 previous similar messages [ 4094.442283] Lustre: Skipped 10 previous similar messages [ 4097.152490] LustreError: 244336:0:(obd_class.h:479:obd_check_dev()) Device 10 not setup [ 4097.154855] LustreError: 244336:0:(obd_class.h:479:obd_check_dev()) Skipped 25 previous similar messages [ 4097.187389] Lustre: server umount lustre-MDT0000 complete [ 4098.569486] LustreError: 242494:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1762170479 with bad export cookie 8117473437442512759 [ 4098.574788] LustreError: 242494:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 4098.574804] LustreError: MGC192.168.202.147@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4099.552687] LustreError: 243284:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4099.553048] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4099.558569] LustreError: 243284:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 4099.566945] Lustre: Skipped 1 previous similar message [ 4104.768239] Lustre: server umount lustre-MDT0001 complete [ 4112.325895] Lustre: server umount lustre-OST0000 complete [ 4128.223130] Lustre: lustre-OST0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 4128.286777] Lustre: server umount lustre-OST0001 complete [ 4130.757885] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_hostid [ 4134.202352] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing load_modules_local [ 4138.121632] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4140.631686] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4142.882215] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4145.169528] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4147.714247] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4147.736618] LDISKFS-fs (dm-0): Mount option "noacl" will be removed by 3.5 [ 4147.736618] Contact linux-ldiskfs@vger.kernel.org if you think we should keep it. [ 4147.736618] [ 4147.744236] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: acl,user_xattr,errors=remount-ro,noacl,no_mbcache,nodelalloc [ 4147.833237] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 4147.847731] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 4147.888602] Lustre: lustre-MDT0000: new disk, initializing [ 4147.915729] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4147.918338] Lustre: Skipped 3 previous similar messages [ 4147.925925] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 4149.349364] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 4153.766147] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4153.793628] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4153.833553] Lustre: 247124:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 4153.859805] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 4153.862018] Lustre: Skipped 1 previous similar message [ 4153.899110] Lustre: lustre-MDT0001: new disk, initializing [ 4153.951855] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 4153.955812] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 4155.371961] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 4159.726304] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4159.750072] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4159.845950] Lustre: lustre-OST0000: new disk, initializing [ 4159.848538] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 4161.216265] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 4161.220251] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 4161.259630] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 4162.022743] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 4166.526828] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4166.555071] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4166.636994] Lustre: lustre-OST0001: new disk, initializing [ 4166.640032] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 4168.206655] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 4168.211569] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 4168.253573] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 4168.882701] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 4173.604230] Lustre: DEBUG MARKER: == conf-sanity test 104b: Mount uses last flock argument ========================================================== 06:49:14 (1762170554) [ 4177.728198] Lustre: DEBUG MARKER: == conf-sanity test 105: check file creation for ro and rw bind mnt pt ========================================================== 06:49:18 (1762170558) [ 4192.224123] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4192.224433] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4192.227127] LustreError: Skipped 1 previous similar message [ 4192.227746] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4192.232135] Lustre: Skipped 1 previous similar message [ 4192.238144] Lustre: Skipped 5 previous similar messages [ 4195.372729] LustreError: 250541:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 4195.375447] LustreError: 250541:0:(obd_class.h:479:obd_check_dev()) Skipped 25 previous similar messages [ 4195.392843] Lustre: server umount lustre-OST0000 complete [ 4198.880811] LustreError: 248659:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4198.885645] LustreError: 248659:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 4211.167192] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 4211.233841] Lustre: server umount lustre-MDT0000 complete [ 4212.469190] LustreError: 247904:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1762170593 with bad export cookie 8117473437442514313 [ 4212.472692] LustreError: MGC192.168.202.147@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4212.473032] LustreError: 247904:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4214.240294] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4214.243199] Lustre: Skipped 3 previous similar messages [ 4218.670107] Lustre: server umount lustre-MDT0001 complete [ 4222.021272] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing unload_modules_local [ 4240.351126] Lustre: lustre-OST0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 4240.409889] Lustre: server umount lustre-OST0001 complete [ 4242.641301] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_hostid [ 4249.116846] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing load_modules_local [ 4252.873662] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4255.146639] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4257.216295] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4259.400016] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4261.770292] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4261.793542] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4261.877252] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 4261.889944] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 4261.930037] Lustre: lustre-MDT0000: new disk, initializing [ 4261.971169] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 4263.503471] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 4267.874110] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4267.897996] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4267.929685] Lustre: 254003:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 4268.028379] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 4269.350342] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 4272.451806] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 4273.633466] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 4276.073049] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4276.098670] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4276.222720] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4276.224690] Lustre: Skipped 5 previous similar messages [ 4278.010080] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 4278.013017] Lustre: Skipped 1 previous similar message [ 4278.014686] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 4278.056387] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 4278.190690] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 4281.387188] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4288.480561] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4288.481355] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4288.484226] LustreError: Skipped 1 previous similar message [ 4288.485052] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4288.488488] Lustre: Skipped 1 previous similar message [ 4288.489917] Lustre: Skipped 13 previous similar messages [ 4289.946989] Lustre: server umount lustre-OST0000 complete [ 4293.600848] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4297.501591] Lustre: server umount lustre-MDT0000 complete [ 4298.720143] LustreError: 254013:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4298.726969] LustreError: 254013:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 5 previous similar messages [ 4298.827503] LustreError: 253995:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1762170680 with bad export cookie 8117473437442515783 [ 4298.831925] LustreError: 253995:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 1 previous similar message [ 4298.832598] LustreError: MGC192.168.202.147@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4298.973178] Lustre: server umount lustre-MDT0001 complete [ 4302.658124] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing unload_modules_local [ 4303.777345] Key type lgssc unregistered [ 4303.918577] LNet: 257376:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4303.921383] LNetError: 257376:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4303.928385] LNet: Removed LNI 192.168.202.147@tcp [ 4304.251173] Key type .llcrypt unregistered [ 4304.252489] Key type ._llcrypt unregistered [ 4314.571566] Key type ._llcrypt registered [ 4314.573263] Key type .llcrypt registered [ 4314.633164] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing load_modules_local [ 4315.013993] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4315.064896] alg: No test for adler32 (adler32-zlib) [ 4315.957722] Lustre: Lustre: Build Version: 2.16.59_50_g4204c43 [ 4316.049046] LNet: Added LNI 192.168.202.147@tcp [8/256/0/180] [ 4317.639207] Key type lgssc registered [ 4318.050621] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4320.788736] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4320.797460] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4321.977297] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4323.333448] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 4326.470805] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4326.610938] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4328.038399] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 4330.081131] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 4331.491794] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 4334.003502] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4334.133101] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4336.225437] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 4338.252171] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4338.606978] Lustre: Setting parameter lustre-OST0000.ost.unknown_param=50 in log lustre-OST0000 [ 4341.217551] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4341.222878] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4341.223306] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4345.325166] LustreError: 261103:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 4345.364351] Lustre: server umount lustre-OST0000 complete [ 4347.360038] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4347.363046] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4347.369598] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4351.456168] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4351.456750] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4356.576103] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4356.578449] Lustre: Skipped 2 previous similar messages [ 4360.673132] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4360.675431] Lustre: Skipped 1 previous similar message [ 4361.183126] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 4361.222720] LustreError: 261308:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup [ 4361.224976] LustreError: 261308:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 4361.248959] Lustre: server umount lustre-MDT0000 complete [ 4362.553170] LustreError: 258794:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1762170743 with bad export cookie 14951895317147050002 [ 4362.554976] LustreError: MGC192.168.202.147@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4362.556243] LustreError: 258794:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 1 previous similar message [ 4362.624619] LustreError: 261513:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [ 4362.627180] LustreError: 261513:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 4362.676889] Lustre: server umount lustre-MDT0001 complete [ 4366.365711] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing unload_modules_local [ 4367.493809] Key type lgssc unregistered [ 4367.638457] LNet: 262043:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4367.640760] LNetError: 262043:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4367.647459] LNet: Removed LNI 192.168.202.147@tcp [ 4367.959987] Key type .llcrypt unregistered [ 4367.961064] Key type ._llcrypt unregistered [ 4374.726516] Key type ._llcrypt registered [ 4374.728239] Key type .llcrypt registered [ 4374.773521] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing load_modules_local [ 4375.077183] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4375.112248] alg: No test for adler32 (adler32-zlib) [ 4375.974754] Lustre: Lustre: Build Version: 2.16.59_50_g4204c43 [ 4376.066025] LNet: Added LNI 192.168.202.147@tcp [8/256/0/180] [ 4377.655187] Key type lgssc registered [ 4378.062502] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4381.322655] Lustre: lustre-OST0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4381.332214] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4396.767325] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 4401.887217] LustreError: 263193:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.202.147@tcp: failed processing log, type 4: rc = -110 [ 4428.511196] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4430.642165] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 4432.744796] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4433.641846] LustreError: 263861:0:(obd_class.h:479:obd_check_dev()) Device 1 not setup [ 4433.693729] Lustre: server umount lustre-OST0000 complete [ 4440.032314] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing unload_modules_local [ 4441.305483] Key type lgssc unregistered [ 4441.450514] LNet: 264696:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4441.453678] LNetError: 264696:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4441.467505] LNet: Removed LNI 192.168.202.147@tcp [ 4441.811729] Key type .llcrypt unregistered [ 4441.814092] Key type ._llcrypt unregistered [ 4458.344754] Key type ._llcrypt registered [ 4458.346239] Key type .llcrypt registered [ 4458.392945] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing load_modules_local [ 4458.767812] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4458.813064] alg: No test for adler32 (adler32-zlib) [ 4459.688561] Lustre: Lustre: Build Version: 2.16.59_50_g4204c43 [ 4459.788204] LNet: Added LNI 192.168.202.147@tcp [8/256/0/180] [ 4461.375267] Key type lgssc registered [ 4461.778217] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4466.878333] loop0: detected capacity change from 0 to 204800000 [ 4466.987682] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 4466.992589] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 4467.127225] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 4467.151993] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4467.520989] loop0: detected capacity change from 0 to 204800000 [ 4467.586051] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 4467.591325] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 4467.734396] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 4467.755696] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4468.168297] loop0: detected capacity change from 0 to 204800000 [ 4468.235130] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 4468.239597] blk_update_request: operation not supported error, dev loop0, sector 8224 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 4468.294877] blk_update_request: operation not supported error, dev loop0, sector 58264 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 4468.347972] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4468.740573] loop0: detected capacity change from 0 to 204800000 [ 4468.812126] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 4468.912536] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4469.304515] loop0: detected capacity change from 0 to 204800000 [ 4469.314239] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 4470.827097] loop0: detected capacity change from 0 to 204800000 [ 4470.842490] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 4472.195457] loop0: detected capacity change from 0 to 204800000 [ 4472.208329] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 4473.599901] loop0: detected capacity change from 0 to 204800000 [ 4473.613918] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 4475.089378] loop0: detected capacity change from 0 to 204800000 [ 4475.127512] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4475.144580] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4475.151791] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4475.180847] Lustre: 268070:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 4475.185902] Lustre: 268070:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index 'nodemap' with [0x200000003:0x3:0x0]: rc = 0 [ 4475.190318] Lustre: 268070:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000003:0xb:0x0]: rc = 0 [ 4475.194825] Lustre: 268070:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x2010000-MDT0000' with [0x200000005:0x3:0x0]: rc = 0 [ 4475.199564] Lustre: 268070:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x10000-MDT0000' with [0x200000005:0x1:0x0]: rc = 0 [ 4475.203754] Lustre: 268070:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000003:0xa:0x0]: rc = 0 [ 4475.208598] Lustre: 268070:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x1010000-MDT0000' with [0x200000005:0x2:0x0]: rc = 0 [ 4475.213232] Lustre: 268070:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000003:0xc:0x0]: rc = 0 [ 4475.217441] Lustre: 268070:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000006:0x1010000:0x0]: rc = 0 [ 4475.221749] Lustre: 268070:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000006:0x10000:0x0]: rc = 0 [ 4475.225665] Lustre: 268070:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000006:0x2010000:0x0]: rc = 0 [ 4475.230515] Lustre: 268070:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x1020000' with [0x200000006:0x1020000:0x0]: rc = 0 [ 4475.236421] Lustre: 268070:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x2020000' with [0x200000006:0x2020000:0x0]: rc = 0 [ 4475.240971] Lustre: 268070:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0000: restore index '0x20000' with [0x200000006:0x20000:0x0]: rc = 0 [ 4478.034859] LustreError: 268338:0:(obd_class.h:479:obd_check_dev()) Device 2 not setup [ 4478.091230] Lustre: server umount MGS complete [ 4478.463114] loop0: detected capacity change from 0 to 204800000 [ 4478.500243] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4478.721824] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4478.744035] Lustre: lustre-MDT0000: Aborting client recovery [ 4479.135229] loop1: detected capacity change from 0 to 204800000 [ 4479.179399] LDISKFS-fs (loop1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4479.227752] LDISKFS-fs (loop1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4479.251285] Lustre: 268507:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 4479.256014] Lustre: 268507:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0001: restore index '0x1010000-MDT0001' with [0x200000005:0x2:0x0]: rc = 0 [ 4479.262235] Lustre: 268507:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0001: restore index '0x2010000-MDT0001' with [0x200000005:0x3:0x0]: rc = 0 [ 4479.267978] Lustre: 268507:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0001: restore index '0x1010000' with [0x200000003:0x5:0x0]: rc = 0 [ 4479.273281] Lustre: 268507:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0001: restore index '0x2010000' with [0x200000003:0x6:0x0]: rc = 0 [ 4479.278547] Lustre: 268507:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0001: restore index '0x10000-MDT0001' with [0x200000005:0x1:0x0]: rc = 0 [ 4479.283582] Lustre: 268507:0:(osd_scrub.c:1842:osd_index_restore()) lustre-MDT0001: restore index '0x10000' with [0x200000003:0x4:0x0]: rc = 0 [ 4479.293109] Lustre: Found index 1 for lustre-MDT0001, updating log [ 4479.378769] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4479.402997] Lustre: lustre-MDT0001: Aborting client recovery [ 4479.772996] loop2: detected capacity change from 0 to 204800000 [ 4479.848358] LDISKFS-fs (loop2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4479.878311] LDISKFS-fs (loop2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4479.899301] Lustre: 268594:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 4479.904321] Lustre: 268594:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0000: restore index '0x1020000-OST0000' with [0x200000005:0x2:0x0]: rc = 0 [ 4479.908931] Lustre: 268594:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0000: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 4479.912764] Lustre: 268594:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0000: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 4479.916659] Lustre: 268594:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0000: restore index '0x2020000-OST0000' with [0x200000005:0x3:0x0]: rc = 0 [ 4479.921348] Lustre: 268594:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0000: restore index '0x20000-OST0000' with [0x200000005:0x1:0x0]: rc = 0 [ 4479.925822] Lustre: 268594:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0000: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 4479.982875] Lustre: Found index 0 for lustre-OST0000, updating log [ 4480.029462] Lustre: lustre-OST0000: Aborting recovery [ 4480.413926] loop3: detected capacity change from 0 to 204800000 [ 4480.459560] LDISKFS-fs (loop3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4480.478793] LDISKFS-fs (loop3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4480.504667] Lustre: 268682:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 4480.509370] Lustre: 268682:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0001: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 4480.514248] Lustre: 268682:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0001: restore index '0x1020000-OST0001' with [0x200000005:0x2:0x0]: rc = 0 [ 4480.520080] Lustre: 268682:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0001: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 4480.523621] Lustre: 268682:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0001: restore index '0x20000-OST0001' with [0x200000005:0x1:0x0]: rc = 0 [ 4480.528576] Lustre: 268682:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0001: restore index '0x2020000-OST0001' with [0x200000005:0x3:0x0]: rc = 0 [ 4480.534679] Lustre: 268682:0:(osd_scrub.c:1842:osd_index_restore()) lustre-OST0001: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 4480.585517] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 4480.588200] Lustre: Skipped 1 previous similar message [ 4480.597982] Lustre: lustre-OST0001: Aborting recovery [ 4485.612225] LustreError: 268450:0:(lod_dev.c:510:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 6, retries 0, failed: rc = -108 [ 4488.674317] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 4488.675541] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 4488.682627] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:5 to 0x280000400:33) [ 4488.686327] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000bd0 [ 4488.686444] Lustre: lustre-OST0001: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:5 to 0x2c0000400:33) [ 4488.692337] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000bd0 [ 4492.863697] LustreError: 268418:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0000: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0000 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0000/identity_upcall, time 413us: rc = -2 [ 4505.056411] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4505.058605] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4505.062681] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4509.153092] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4509.153283] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4509.158186] Lustre: Skipped 2 previous similar messages [ 4509.162844] Lustre: Skipped 3 previous similar messages [ 4510.332524] LustreError: 269039:0:(obd_class.h:479:obd_check_dev()) Device 10 not setup [ 4510.334545] LustreError: 269039:0:(obd_class.h:479:obd_check_dev()) Skipped 1 previous similar message [ 4510.369635] Lustre: server umount lustre-MDT0000 complete [ 4510.745215] LustreError: 268517:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1762170892 with bad export cookie 13415467006991880759 [ 4510.747271] LustreError: MGC192.168.202.147@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4510.750521] LustreError: 268517:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4514.271784] LustreError: 268414:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4514.273306] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4514.274167] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 4514.278345] LustreError: 268414:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 4514.283374] Lustre: Skipped 1 previous similar message [ 4515.296446] LustreError: 268414:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4515.302157] LustreError: 268414:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 4516.998217] LustreError: 269091:0:(obd_class.h:479:obd_check_dev()) Device 18 not setup [ 4517.001211] LustreError: 269091:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 4517.089688] Lustre: server umount lustre-MDT0001 complete [ 4523.567088] LustreError: 269142:0:(obd_class.h:479:obd_check_dev()) Device 23 not setup [ 4523.569864] LustreError: 269142:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 4523.603972] Lustre: server umount lustre-OST0000 complete [ 4538.335146] Lustre: lustre-OST0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 4538.352532] LustreError: 269193:0:(obd_class.h:479:obd_check_dev()) Device 2 not setup [ 4538.355017] LustreError: 269193:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 4538.411658] Lustre: server umount lustre-OST0001 complete [ 4541.228682] Lustre: DEBUG MARKER: == conf-sanity test 109a: test lctl clear_conf fsname ==== 06:55:22 (1762170922) [ 4550.420035] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_hostid [ 4553.548425] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing load_modules_local [ 4557.127484] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4559.649706] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4561.965750] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4564.265054] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4566.807071] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4566.833649] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4566.923036] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 4566.935591] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 4566.975709] Lustre: lustre-MDT0000: new disk, initializing [ 4567.003318] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4567.010802] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 4568.408213] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 4572.812204] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4572.837357] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4572.872967] Lustre: 272738:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 4572.898413] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 4572.900725] Lustre: Skipped 1 previous similar message [ 4572.938580] Lustre: lustre-MDT0001: new disk, initializing [ 4572.975053] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4572.988239] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 4572.991229] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 4574.470905] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 4577.733427] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 4578.999633] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 4581.646638] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4581.681839] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4581.789148] Lustre: lustre-OST0000: new disk, initializing [ 4581.791746] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 4581.827645] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4584.051076] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 4587.516441] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4590.073090] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 4590.076909] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 4590.112340] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 4595.212258] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [ 4599.325509] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [ 4604.063953] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [ 4646.369951] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4646.370652] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4646.372362] LustreError: Skipped 1 previous similar message [ 4646.372963] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4646.376094] Lustre: Skipped 1 previous similar message [ 4646.377453] Lustre: Skipped 3 previous similar messages [ 4650.669402] LustreError: 276204:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 4650.671466] LustreError: 276204:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 4650.715058] Lustre: server umount lustre-OST0000 complete [ 4655.071872] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4655.073985] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4655.078440] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4655.079853] Lustre: Skipped 1 previous similar message [ 4665.824196] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4665.826632] Lustre: Skipped 5 previous similar messages [ 4666.335139] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 4666.401172] Lustre: server umount lustre-MDT0000 complete [ 4667.715499] LustreError: 272729:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1762171049 with bad export cookie 13415467006991883461 [ 4667.718358] LustreError: MGC192.168.202.147@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4667.720088] LustreError: 272729:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4667.789851] LustreError: 276608:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 4667.792556] LustreError: 276608:0:(obd_class.h:479:obd_check_dev()) Skipped 9 previous similar messages [ 4667.851149] Lustre: server umount lustre-MDT0001 complete [ 4670.991191] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4672.355729] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 4674.060448] Lustre: server umount MGS complete [ 4677.889205] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4678.044125] LustreError: 278171:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4678.052121] LustreError: 278183:0:(qmt_pool.c:1523:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [ 4678.074835] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4679.336793] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 4682.298843] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4683.844231] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 4685.883329] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 4687.208173] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 4689.657342] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4691.807955] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 4693.659637] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4724.708679] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4724.708896] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4724.709737] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4724.717593] Lustre: Skipped 2 previous similar messages [ 4729.389866] LustreError: 280851:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 4729.391516] LustreError: 280851:0:(obd_class.h:479:obd_check_dev()) Skipped 9 previous similar messages [ 4729.427626] Lustre: server umount lustre-OST0000 complete [ 4733.919819] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4736.856311] Lustre: server umount lustre-MDT0000 complete [ 4738.116447] LustreError: 278150:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1762171119 with bad export cookie 13415467006991885288 [ 4738.116856] LustreError: MGC192.168.202.147@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4738.122087] LustreError: 278150:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 4738.239531] Lustre: server umount lustre-MDT0001 complete [ 4741.718875] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing unload_modules_local [ 4742.768559] Key type lgssc unregistered [ 4742.900374] LNet: 281787:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4742.902586] LNetError: 281787:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4742.911339] LNet: Removed LNI 192.168.202.147@tcp [ 4743.243818] Key type .llcrypt unregistered [ 4743.245603] Key type ._llcrypt unregistered [ 4756.210943] Key type ._llcrypt registered [ 4756.212128] Key type .llcrypt registered [ 4756.247188] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_hostid [ 4762.586382] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing load_modules_local [ 4762.925477] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4762.931919] alg: No test for adler32 (adler32-zlib) [ 4763.794414] Lustre: Lustre: Build Version: 2.16.59_50_g4204c43 [ 4763.875089] LNet: Added LNI 192.168.202.147@tcp [8/256/0/180] [ 4765.455136] Key type lgssc registered [ 4765.836387] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4768.744712] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4770.973941] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4773.001851] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4775.234684] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4777.600867] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4777.622269] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4777.627183] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4778.725667] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 4778.737930] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 4778.778183] Lustre: lustre-MDT0000: new disk, initializing [ 4778.811732] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4778.820236] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 4780.148959] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 4784.385039] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4784.411799] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4784.444817] Lustre: 285708:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 4784.473745] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 4784.475537] Lustre: Skipped 1 previous similar message [ 4784.511019] Lustre: lustre-MDT0001: new disk, initializing [ 4784.549080] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4784.563694] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 4784.568826] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 4785.878106] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 4788.979707] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 4790.171505] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 4792.456196] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4792.478583] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4792.566776] Lustre: lustre-OST0000: new disk, initializing [ 4792.569619] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 4792.592833] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4794.588580] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 4796.919109] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 4796.923052] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 4796.955887] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 4797.671534] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4801.092871] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [ 4807.448453] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [ 4814.569849] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [ 4822.126510] Lustre: Modifying parameter lustre-client.llite.max_read_ahead_mb=64 in log lustre-client [ 4858.336854] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4858.337734] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4858.339048] LustreError: Skipped 1 previous similar message [ 4858.339341] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4858.344467] Lustre: Skipped 1 previous similar message [ 4863.457996] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4863.461140] Lustre: Skipped 2 previous similar messages [ 4864.174145] LustreError: 289371:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 4864.226285] Lustre: server umount lustre-OST0000 complete [ 4866.528146] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4866.530957] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4866.535300] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4868.576533] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4868.576961] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4871.747111] LustreError: 289572:0:(obd_class.h:479:obd_check_dev()) Device 22 not setup [ 4871.750246] LustreError: 289572:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 4871.788581] Lustre: server umount lustre-MDT0000 complete [ 4873.156958] LustreError: 288936:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1762171254 with bad export cookie 16833895203605098825 [ 4873.159804] LustreError: MGC192.168.202.147@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4873.160338] LustreError: 288936:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4873.229604] LustreError: 289773:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 4873.232019] LustreError: 289773:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 4873.290663] Lustre: server umount lustre-MDT0001 complete [ 4876.083502] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4877.311776] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 4878.904562] LustreError: 290714:0:(obd_class.h:479:obd_check_dev()) Device 2 not setup [ 4878.907021] LustreError: 290714:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 4878.928813] Lustre: server umount MGS complete [ 4882.593518] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4882.749568] LustreError: 291359:0:(qmt_pool.c:1523:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [ 4882.772247] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4884.037414] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 4887.021805] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4887.182398] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4888.489239] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 4890.184798] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 4891.301480] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 4893.560801] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4895.496736] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 4897.257426] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4912.100850] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4912.100953] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4912.102240] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4912.102248] Lustre: Skipped 1 previous similar message [ 4912.106107] Lustre: Skipped 1 previous similar message [ 4918.123707] LustreError: 294001:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 4918.125680] LustreError: 294001:0:(obd_class.h:479:obd_check_dev()) Skipped 1 previous similar message [ 4918.163313] Lustre: server umount lustre-OST0000 complete [ 4922.336877] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4922.337618] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4922.340618] Lustre: Skipped 1 previous similar message [ 4922.343292] Lustre: Skipped 4 previous similar messages [ 4925.530740] Lustre: server umount lustre-MDT0000 complete [ 4926.841703] LustreError: 293738:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1762171308 with bad export cookie 16833895203605100750 [ 4926.844827] LustreError: MGC192.168.202.147@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4926.846718] LustreError: 293738:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4926.915654] LustreError: 294405:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [ 4926.918084] LustreError: 294405:0:(obd_class.h:479:obd_check_dev()) Skipped 9 previous similar messages [ 4926.973984] Lustre: server umount lustre-MDT0001 complete [ 4930.741262] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing unload_modules_local [ 4931.818440] Key type lgssc unregistered [ 4931.946362] LNet: 294936:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4931.950257] LNetError: 294936:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4931.960291] LNet: Removed LNI 192.168.202.147@tcp [ 4932.264872] Key type .llcrypt unregistered [ 4932.265836] Key type ._llcrypt unregistered [ 4939.190336] Key type ._llcrypt registered [ 4939.192150] Key type .llcrypt registered [ 4939.231570] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_hostid [ 4945.207377] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing load_modules_local [ 4945.645740] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4945.682433] alg: No test for adler32 (adler32-zlib) [ 4946.572083] Lustre: Lustre: Build Version: 2.16.59_50_g4204c43 [ 4946.676831] LNet: Added LNI 192.168.202.147@tcp [8/256/0/180] [ 4948.271228] Key type lgssc registered [ 4948.731257] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4952.017543] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4954.181258] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4957.273837] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing load_modules_local [ 4960.255817] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4962.672153] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4962.689946] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4962.694765] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4963.785049] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 4963.800862] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 4963.846341] Lustre: lustre-MDT0000: new disk, initializing [ 4963.877689] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4963.885211] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 4965.353687] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 4969.578085] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4969.603630] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 4969.710402] Lustre: lustre-OST0000: new disk, initializing [ 4969.712183] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 4969.713631] Lustre: Skipped 1 previous similar message [ 4969.738181] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4971.854551] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 4971.979696] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 4971.982662] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 4971.991810] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 5151.200509] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5151.205516] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5156.044415] LustreError: 299844:0:(obd_class.h:479:obd_check_dev()) Device 13 not setup [ 5156.139768] Lustre: server umount lustre-MDT0000 complete [ 5157.398466] LustreError: 298097:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1762171538 with bad export cookie 16837445864980579442 [ 5157.400775] LustreError: MGC192.168.202.147@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5157.401679] LustreError: 298097:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 5157.424315] LustreError: 300048:0:(obd_class.h:479:obd_check_dev()) Device 2 not setup [ 5157.426407] LustreError: 300048:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 5157.467437] Lustre: server umount lustre-OST0000 complete [ 5165.526430] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_hostid [ 5168.504229] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing load_modules_local [ 5171.996239] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5173.989303] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5176.070864] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5178.344776] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5180.666391] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5180.692858] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5180.788409] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 5180.800481] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 5180.842669] Lustre: lustre-MDT0000: new disk, initializing [ 5180.870388] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5180.876774] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 5182.251570] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 5186.499137] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5186.523408] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5186.561132] Lustre: 303085:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 5186.585145] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 5186.587027] Lustre: Skipped 1 previous similar message [ 5186.627116] Lustre: lustre-MDT0001: new disk, initializing [ 5186.659147] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5186.672499] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 5186.675735] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 5187.992124] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 5191.184980] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 5192.452283] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 5194.874762] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5194.897580] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5195.002101] Lustre: lustre-OST0000: new disk, initializing [ 5195.004919] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 5195.042264] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5195.088542] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 5195.095456] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 5195.144663] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 5197.194507] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 5200.521135] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 5201.962314] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 5202.056576] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 5203.504745] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 5203.597497] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 5205.473953] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 5205.474665] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5205.475307] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5205.476265] LustreError: Skipped 1 previous similar message [ 5205.479398] Lustre: Skipped 1 previous similar message [ 5206.318078] LustreError: 306021:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 5206.320196] LustreError: 306021:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 5206.362130] Lustre: server umount lustre-OST0000 complete [ 5210.592895] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5210.593541] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5210.596625] Lustre: Skipped 1 previous similar message [ 5210.598572] Lustre: Skipped 1 previous similar message [ 5213.823851] LustreError: 306223:0:(obd_class.h:479:obd_check_dev()) Device 22 not setup [ 5213.826536] LustreError: 306223:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 5213.855230] Lustre: server umount lustre-MDT0000 complete [ 5215.201289] LustreError: 303076:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1762171596 with bad export cookie 16837445864983520219 [ 5215.204744] LustreError: 303076:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 1 previous similar message [ 5215.206085] LustreError: MGC192.168.202.147@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5215.332103] Lustre: server umount lustre-MDT0001 complete [ 5216.619970] Lustre: DEBUG MARKER: SKIP: conf-sanity test_111 skipping SLOW test 111 [ 5217.268585] Lustre: DEBUG MARKER: == conf-sanity test 112a: mount OST with no_create option ========================================================== 07:06:38 (1762171598) [ 5219.688323] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5219.885949] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5221.329199] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 5224.437610] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5225.912789] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 5227.838970] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 5229.069560] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 5231.376900] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5233.528801] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 5235.387787] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 5237.751976] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5237.776414] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5237.838813] Lustre: lustre-OST0001: new disk, initializing [ 5237.840885] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 5237.879676] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 5237.881566] Lustre: Skipped 2 previous similar messages [ 5239.876037] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 5239.879838] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 5239.900670] LustreError: 309802:0:(osp_precreate.c:969:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [ 5240.021573] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 5240.931211] LustreError: 309802:0:(osp_precreate.c:969:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [ 5240.934108] LustreError: 309802:0:(osp_precreate.c:969:osp_precreate_cleanup_orphans()) Skipped 2 previous similar messages [ 5241.954175] LustreError: 309802:0:(osp_precreate.c:969:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [ 5241.957481] LustreError: 309802:0:(osp_precreate.c:969:osp_precreate_cleanup_orphans()) Skipped 1 previous similar message [ 5243.904183] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 5244.003163] LustreError: 309809:0:(osp_precreate.c:969:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [ 5244.006519] LustreError: 309809:0:(osp_precreate.c:969:osp_precreate_cleanup_orphans()) Skipped 3 previous similar messages [ 5246.042149] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 2 sec [ 5247.389477] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff94f444f33000.ost_server_uuid 50 [ 5247.936397] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff94f444f33000.ost_server_uuid in FULL state after 0 sec [ 5248.099347] LustreError: 309809:0:(osp_precreate.c:969:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [ 5248.102507] LustreError: 309809:0:(osp_precreate.c:969:osp_precreate_cleanup_orphans()) Skipped 7 previous similar messages [ 5249.266371] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff94f444f33000.ost_server_uuid 50 [ 5249.811855] Lustre: DEBUG MARKER: osc.lustre-OST0001-osc-ffff94f444f33000.ost_server_uuid in FULL state after 0 sec [ 5251.176540] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 5261.792907] LustreError: lustre-OST0001-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 5261.792942] Lustre: lustre-OST0001-osc-MDT0001: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5261.796540] LustreError: Skipped 1 previous similar message [ 5261.801512] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 5261.803613] Lustre: Skipped 2 previous similar messages [ 5265.896491] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 5265.899224] Lustre: Skipped 2 previous similar messages [ 5267.632133] LustreError: 310820:0:(obd_class.h:479:obd_check_dev()) Device 25 not setup [ 5267.635180] LustreError: 310820:0:(obd_class.h:479:obd_check_dev()) Skipped 13 previous similar messages [ 5267.656866] Lustre: server umount lustre-OST0001 complete [ 5268.047419] LustreError: 310472:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 192.168.202.47@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5268.055918] LustreError: 310472:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 5270.495966] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 5270.498364] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5270.502554] Lustre: Skipped 1 previous similar message [ 5271.013523] LustreError: 310472:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5271.021799] LustreError: 310472:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 5275.404386] Lustre: server umount lustre-OST0000 complete [ 5281.248614] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 5281.248786] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5281.249681] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5281.249687] Lustre: Skipped 3 previous similar messages [ 5281.251559] LustreError: Skipped 2 previous similar messages [ 5281.255485] Lustre: Skipped 2 previous similar messages [ 5290.975122] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 5291.008320] LustreError: 311222:0:(obd_class.h:479:obd_check_dev()) Device 28 not setup [ 5291.010535] LustreError: 311222:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 5291.037670] Lustre: server umount lustre-MDT0000 complete [ 5291.488344] LustreError: 306996:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5291.495934] LustreError: 306996:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 5292.266168] LustreError: 306978:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1762171673 with bad export cookie 16837445864983521087 [ 5292.270909] LustreError: MGC192.168.202.147@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5292.390892] Lustre: server umount lustre-MDT0001 complete [ 5295.751205] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing unload_modules_local [ 5296.716048] Key type lgssc unregistered [ 5296.838340] LNet: 311955:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5296.841129] LNetError: 311955:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5296.850308] LNet: Removed LNI 192.168.202.147@tcp [ 5297.123734] Key type .llcrypt unregistered [ 5297.125070] Key type ._llcrypt unregistered [ 5306.016555] Key type ._llcrypt registered [ 5306.017482] Key type .llcrypt registered [ 5306.058953] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing load_modules_local [ 5306.404617] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5306.427749] alg: No test for adler32 (adler32-zlib) [ 5307.285906] Lustre: Lustre: Build Version: 2.16.59_50_g4204c43 [ 5307.368936] LNet: Added LNI 192.168.202.147@tcp [8/256/0/180] [ 5308.951158] Key type lgssc registered [ 5309.329431] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5311.921776] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5311.927592] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5313.081465] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5314.334574] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 5315.999823] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 5318.177910] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5318.300621] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5319.514501] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 5322.435642] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5322.556804] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5323.620402] LustreError: 314885:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5323.624638] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:65) [ 5323.627025] LustreError: 314885:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 5324.507563] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 5326.367303] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 5327.717979] LustreError: 314884:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5327.724687] LustreError: 314884:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 5328.917819] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5329.000504] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 5329.276276] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:33) [ 5331.074647] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 5332.975543] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid [ 5335.617303] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing wait_import_state (FULL|IDLE) os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [ 5335.703787] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 5416.420659] LustreError: lustre-OST0001-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 5416.422396] Lustre: lustre-OST0001-osc-MDT0000: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5416.435398] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 5416.438655] Lustre: Skipped 1 previous similar message [ 5418.285839] LustreError: 316757:0:(obd_class.h:479:obd_check_dev()) Device 27 not setup [ 5418.302872] Lustre: server umount lustre-OST0001 complete [ 5420.000231] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 5420.002602] LustreError: Skipped 1 previous similar message [ 5420.003767] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5420.007618] Lustre: Skipped 1 previous similar message [ 5420.009820] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5421.025374] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 5421.028225] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5421.032408] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5421.537154] LustreError: 314885:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5421.542243] LustreError: 314885:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 5426.657109] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5426.657219] LustreError: 315602:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5426.658874] Lustre: Skipped 2 previous similar messages [ 5426.663925] LustreError: 315602:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 5431.777730] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5431.779141] LustreError: 315070:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5431.780513] Lustre: Skipped 1 previous similar message [ 5431.786047] LustreError: 315070:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 5434.335143] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 5434.355105] LustreError: 316958:0:(obd_class.h:479:obd_check_dev()) Device 23 not setup [ 5434.358027] LustreError: 316958:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 5434.398890] Lustre: server umount lustre-OST0000 complete [ 5436.384170] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 5436.386256] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5441.793839] LustreError: 317160:0:(obd_class.h:479:obd_check_dev()) Device 10 not setup [ 5441.795849] LustreError: 317160:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 5441.829243] Lustre: server umount lustre-MDT0000 complete [ 5442.017169] LustreError: 314903:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5442.024103] LustreError: 314903:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 5443.217362] LustreError: 317162:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1762171824 with bad export cookie 18055692378500458537 [ 5443.220886] LustreError: MGC192.168.202.147@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5443.221447] LustreError: 317162:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 5443.364465] Lustre: server umount lustre-MDT0001 complete [ 5446.987817] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing unload_modules_local [ 5448.008237] Key type lgssc unregistered [ 5448.131726] LNet: 317894:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5448.134830] LNetError: 317894:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5448.142374] LNet: Removed LNI 192.168.202.147@tcp [ 5448.429423] Key type .llcrypt unregistered [ 5448.430922] Key type ._llcrypt unregistered [ 5465.597081] Key type ._llcrypt registered [ 5465.598135] Key type .llcrypt registered [ 5465.631619] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing load_modules_local [ 5465.937576] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5465.946919] alg: No test for adler32 (adler32-zlib) [ 5466.801688] Lustre: Lustre: Build Version: 2.16.59_50_g4204c43 [ 5466.885508] LNet: Added LNI 192.168.202.147@tcp [8/256/0/180] [ 5468.463137] Key type lgssc registered [ 5468.868197] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5472.169405] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5472.176816] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5473.345487] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5474.676591] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 5477.644261] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5477.781784] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5479.093683] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 5480.078299] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 5482.362293] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5482.486322] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5483.490022] LustreError: 321602:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5483.494466] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:97) [ 5483.494845] LustreError: 321602:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 5484.387178] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 5487.339290] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5487.417251] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 5488.428585] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:65) [ 5489.402694] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 5493.638475] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 5501.465749] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 5513.695813] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 5513.698043] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5513.701837] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5514.211703] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5514.211791] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5514.214942] Lustre: Skipped 1 previous similar message [ 5514.220431] Lustre: Skipped 3 previous similar messages [ 5516.291743] LustreError: 323553:0:(obd_class.h:479:obd_check_dev()) Device 10 not setup [ 5516.331094] Lustre: server umount lustre-MDT0000 complete [ 5517.611803] LustreError: 320120:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1762171898 with bad export cookie 5601373502777136170 [ 5517.617452] LustreError: MGC192.168.202.147@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5517.689654] LustreError: 323754:0:(obd_class.h:479:obd_check_dev()) Device 18 not setup [ 5517.691493] LustreError: 323754:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 5517.734308] Lustre: server umount lustre-MDT0001 complete [ 5529.525089] LustreError: 323954:0:(obd_class.h:479:obd_check_dev()) Device 23 not setup [ 5529.527346] LustreError: 323954:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages [ 5529.548115] Lustre: server umount lustre-OST0000 complete [ 5541.106159] LustreError: 324156:0:(obd_class.h:479:obd_check_dev()) Device 2 not setup [ 5541.108843] LustreError: 324156:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages [ 5541.158573] Lustre: server umount lustre-OST0001 complete [ 5546.589212] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing load_modules_local [ 5550.278301] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5550.451446] LustreError: 325331:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5550.474660] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5551.839372] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 5554.752704] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5556.257293] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 5557.294062] Lustre: 326440:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 5559.610146] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5559.743815] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5559.745729] Lustre: Skipped 1 previous similar message [ 5560.804279] LustreError: 326796:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5560.807468] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:129) [ 5561.805638] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 5564.759911] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5565.872146] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:97) [ 5566.940212] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 5570.175367] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 5571.384539] Lustre: 328285:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 5580.767921] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 5580.770585] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5580.774875] Lustre: Skipped 1 previous similar message [ 5580.776981] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5583.489403] LustreError: 328390:0:(obd_class.h:479:obd_check_dev()) Device 10 not setup [ 5583.492565] LustreError: 328390:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 5583.527329] Lustre: server umount lustre-MDT0000 complete [ 5584.754250] LustreError: 328287:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1762171966 with bad export cookie 5601373502777137829 [ 5584.757627] LustreError: MGC192.168.202.147@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5584.759463] LustreError: 328287:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 7 previous similar messages [ 5584.874678] Lustre: server umount lustre-MDT0001 complete [ 5596.592098] LustreError: 328794:0:(obd_class.h:479:obd_check_dev()) Device 23 not setup [ 5596.594612] LustreError: 328794:0:(obd_class.h:479:obd_check_dev()) Skipped 15 previous similar messages [ 5596.609711] Lustre: server umount lustre-OST0000 complete [ 5608.086088] Lustre: server umount lustre-OST0001 complete [ 5609.176177] Lustre: DEBUG MARKER: SKIP: conf-sanity test_114 skipping SLOW test 114 [ 5609.696986] Lustre: DEBUG MARKER: SKIP: conf-sanity test_115 skipping excluded test 115 [ 5610.201460] Lustre: DEBUG MARKER: == conf-sanity test 116: big size MDT support ============ 07:13:11 (1762171991) [ 5616.987244] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing load_modules_local [ 5619.778480] loop0: detected capacity change from 0 to 1099511627776 [ 5619.902373] SGI XFS with ACLs, security attributes, quota, no debug enabled [ 5619.904493] *** VALIDATE xfs *** [ 5619.907891] XFS (loop0): Mounting V5 Filesystem [ 5619.916527] XFS (loop0): Ending clean mount [ 5620.294595] loop1: detected capacity change from 0 to 18691697672192 [ 5634.379047] LDISKFS-fs (loop1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5640.173703] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_hostid [ 5643.285361] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing load_modules_local [ 5646.602818] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5648.702771] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5650.712709] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5652.849757] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5655.056104] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5655.077553] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5655.158769] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 5655.167948] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 5655.201587] Lustre: lustre-MDT0000: new disk, initializing [ 5655.223959] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5655.225632] Lustre: Skipped 1 previous similar message [ 5655.230653] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 5656.540455] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 5660.637667] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5660.665908] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5660.701609] Lustre: 333367:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 5660.724067] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 5660.725871] Lustre: Skipped 1 previous similar message [ 5660.759942] Lustre: lustre-MDT0001: new disk, initializing [ 5660.806153] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 5660.810177] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 5662.126738] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 5665.330795] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 5666.604992] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 5668.923017] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5668.945463] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5669.042485] Lustre: lustre-OST0000: new disk, initializing [ 5669.044032] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 5670.893015] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 5670.976271] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 5670.980890] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 5671.016050] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 5674.166242] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 5675.678770] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 5675.762418] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 5677.065183] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 5677.132159] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 5679.010781] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5679.011549] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5679.016367] Lustre: Skipped 4 previous similar messages [ 5679.017791] Lustre: Skipped 4 previous similar messages [ 5679.854091] LustreError: 336302:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup [ 5679.855915] LustreError: 336302:0:(obd_class.h:479:obd_check_dev()) Skipped 9 previous similar messages [ 5679.894543] Lustre: server umount lustre-OST0000 complete [ 5681.631797] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 5684.192169] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5684.192908] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5684.193775] Lustre: Skipped 2 previous similar messages [ 5684.196911] Lustre: Skipped 1 previous similar message [ 5687.271612] Lustre: server umount lustre-MDT0000 complete [ 5688.524308] LustreError: 333359:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1762172069 with bad export cookie 5601373502777139383 [ 5688.527763] LustreError: MGC192.168.202.147@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5688.527816] LustreError: 333359:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 5688.647912] Lustre: server umount lustre-MDT0001 complete [ 5691.377865] XFS (loop0): Unmounting Filesystem [ 5692.969787] Lustre: DEBUG MARKER: == conf-sanity test 117: lctl get_param return errors properly ========================================================== 07:14:34 (1762172074) [ 5695.288929] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5695.442111] LustreError: 337522:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5695.465793] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5695.467754] Lustre: Skipped 2 previous similar messages [ 5696.809067] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 5699.706090] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5701.217723] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 5703.072208] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 5704.322076] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 5706.540435] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5708.503383] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 5710.431470] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 5725.153269] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 5725.155278] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5725.156214] LustreError: Skipped 1 previous similar message [ 5725.159799] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5725.162973] Lustre: Skipped 1 previous similar message [ 5726.253146] LustreError: 339883:0:(obd_class.h:479:obd_check_dev()) Device 21 not setup [ 5726.255964] LustreError: 339883:0:(obd_class.h:479:obd_check_dev()) Skipped 17 previous similar messages [ 5726.290889] Lustre: server umount lustre-OST0000 complete [ 5729.625331] Lustre: server umount lustre-MDT0000 complete [ 5730.274813] LustreError: 337521:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5730.282214] LustreError: 337521:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 5730.935250] LustreError: 340085:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1762172112 with bad export cookie 5601373502777140265 [ 5730.938367] LustreError: 340085:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 1 previous similar message [ 5730.939351] LustreError: MGC192.168.202.147@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5731.063421] Lustre: server umount lustre-MDT0001 complete [ 5734.669823] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing unload_modules_local [ 5735.796422] Key type lgssc unregistered [ 5735.930482] LNet: 340816:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5735.933816] LNetError: 340816:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5735.946304] LNet: Removed LNI 192.168.202.147@tcp [ 5736.279108] Key type .llcrypt unregistered [ 5736.280399] Key type ._llcrypt unregistered [ 5748.581238] Key type ._llcrypt registered [ 5748.582807] Key type .llcrypt registered [ 5748.668732] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing load_modules_local [ 5749.401258] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5749.524367] alg: No test for adler32 (adler32-zlib) [ 5750.548939] Lustre: Lustre: Build Version: 2.16.59_50_g4204c43 [ 5750.780558] LNet: Added LNI 192.168.202.147@tcp [8/256/0/180] [ 5752.431166] Key type lgssc registered [ 5753.456238] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5758.843838] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5758.857196] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5760.170644] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5762.773790] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 5769.282843] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5769.508576] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5771.821968] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 5774.028592] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 5775.341822] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 5777.739131] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5777.863590] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5779.782829] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 5781.611234] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 5790.181028] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5790.187038] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 5790.189198] Lustre: Skipped 1 previous similar message [ 5792.341397] Lustre: lustre-MDT0001: Not available for connect from 192.168.202.47@tcp (stopping) [ 5795.297358] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 5795.299826] Lustre: Skipped 1 previous similar message [ 5795.586307] LustreError: 344638:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [ 5795.615963] Lustre: server umount lustre-MDT0001 complete [ 5796.559208] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5797.454276] LustreError: 342337:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.202.47@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5797.459200] LustreError: 342337:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 5799.001945] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 5799.030901] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [ 5799.032619] Lustre: Found index 1 for lustre-MDT0001, updating log [ 5799.044023] Lustre: 342329:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 5799.155971] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5800.423253] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 5804.524346] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 5805.030107] LustreError: 345718:0:(obd_mount.c:188:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [ 5805.033154] LustreError: 345718:0:(tgt_mount.c:669:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [ 5805.036455] LustreError: 342334:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [ 5805.055746] LustreError: 345721:0:(obd_config.c:1997:class_config_llog_handler()) MGC192.168.202.147@tcp: cfg command failed: rc = -17 [ 5805.058973] Lustre: cmd=cf001 0:lustre-MDT0001-osp-MDT0000 1:osp 2:lustre-MDT0000-mdtlov_UUID [ 5805.058973] [ 6104.660696] Lustre: lustre-MDT0001: Not available for connect from 192.168.202.47@tcp (stopping) [ 6106.593456] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [ 6106.593590] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6106.601694] Lustre: Skipped 1 previous similar message [ 6109.434347] LustreError: 356757:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [ 6109.436487] LustreError: 356757:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 6109.460238] Lustre: server umount lustre-MDT0001 complete [ 6109.773907] LustreError: 344496:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.202.47@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6110.566745] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6111.712514] LustreError: 342341:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6111.719148] LustreError: 342341:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 6113.257985] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 6113.305215] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [ 6113.307229] Lustre: Found index 1 for lustre-MDT0001, updating log [ 6113.321308] Lustre: 342329:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 6113.439874] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 6114.794386] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 6118.888594] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 6120.421538] LustreError: 357883:0:(obd_mount.c:188:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [ 6120.424430] LustreError: 357883:0:(tgt_mount.c:669:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [ 6120.426828] LustreError: 342334:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [ 6120.429397] LustreError: 342334:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [ 6120.446641] LustreError: 357886:0:(obd_config.c:689:class_setup()) Device 8 already setup (type osp) [ 6120.449819] LustreError: 357886:0:(obd_config.c:1997:class_config_llog_handler()) MGC192.168.202.147@tcp: cfg command failed: rc = -17 [ 6120.452683] Lustre: cmd=cf003 0:lustre-MDT0001-osp-MDT0000 1:lustre-MDT0001_UUID 2:192.168.202.147@tcp [ 6120.452683] [ 6418.913106] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6418.918387] Lustre: Skipped 1 previous similar message [ 6418.920382] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 6418.922386] Lustre: Skipped 3 previous similar messages [ 6423.741188] LustreError: 369121:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [ 6423.743828] LustreError: 369121:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 6423.768339] Lustre: server umount lustre-MDT0001 complete [ 6424.033562] LustreError: 366272:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6424.038872] LustreError: 366272:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 6424.715345] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6427.228198] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 6427.263914] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [ 6427.265580] Lustre: Found index 1 for lustre-MDT0001, updating log [ 6427.280180] Lustre: 342329:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 6427.397468] Lustre: lustre-MDT0001: Imperative Recovery enabled, recovery window shrunk from 60-180 down to 60-180 [ 6428.771970] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 6432.744949] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 6432.747113] LustreError: 370198:0:(obd_mount.c:188:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [ 6432.751168] LustreError: 370198:0:(tgt_mount.c:669:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [ 6432.755339] LustreError: 342334:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [ 6432.758093] LustreError: 342334:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [ 6432.776323] LustreError: 370201:0:(lod_lov.c:207:lod_add_device()) lustre-MDT0001-osp-MDT0000: cannot connect to next dev lustre-MDT0001_UUID (-114) [ 6432.782086] LustreError: 370201:0:(obd_config.c:1997:class_config_llog_handler()) MGC192.168.202.147@tcp: cfg command failed: rc = -114 [ 6432.785744] Lustre: cmd=cf014 0:lustre-MDT0000-mdtlov 1:lustre-MDT0001_UUID 2:1 3:1 [ 6432.785744] [ 6745.057743] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6745.059055] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 6745.059885] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6745.059893] Lustre: Skipped 1 previous similar message [ 6745.062529] Lustre: Skipped 3 previous similar messages [ 6749.503850] LustreError: 381855:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup [ 6749.507623] LustreError: 381855:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 6749.544194] Lustre: server umount lustre-MDT0000 complete [ 6750.176470] LustreError: 342335:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6750.181496] LustreError: 342335:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 6750.796625] LustreError: 345205:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1762173132 with bad export cookie 4952169246714609313 [ 6750.800337] LustreError: MGC192.168.202.147@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6750.802100] LustreError: 345205:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 6755.296698] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6756.925125] LustreError: 382056:0:(obd_class.h:479:obd_check_dev()) Device 16 not setup [ 6756.927978] LustreError: 382056:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages [ 6756.982625] Lustre: server umount lustre-MDT0001 complete [ 6758.332095] Lustre: server umount lustre-OST0000 complete [ 6761.433259] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_hostid [ 6764.605840] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing load_modules_local [ 6768.137312] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6770.291630] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6772.384370] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6774.479847] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6776.648327] Lustre: DEBUG MARKER: == conf-sanity test 120: cross-target rename should not create bad symlinks ========================================================== 07:32:37 (1762173157) [ 6778.894066] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6778.917869] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6778.997268] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 6779.007345] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 6779.042288] Lustre: lustre-MDT0000: new disk, initializing [ 6779.066089] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6779.072757] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 6780.468930] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 6784.702349] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6784.725993] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 6784.762390] Lustre: 384803:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 6784.789518] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 6784.792319] Lustre: Skipped 1 previous similar message [ 6784.830315] Lustre: lustre-MDT0001: new disk, initializing [ 6784.863790] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 6784.876096] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 6784.879504] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 6786.182281] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 6789.391635] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 6790.614047] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 6792.962085] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6792.986351] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc [ 6793.091760] Lustre: lustre-OST0000: new disk, initializing [ 6793.094935] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 6793.127128] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 6795.113870] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_default_debug -1 all [ 6795.768125] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 6795.772728] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 6795.805907] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 6798.260996] Lustre: DEBUG MARKER: oleg247-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 6799.829971] Lustre: 387155:0:(osd_internal.h:1471:osd_trans_exec_op()) lustre-MDT0001: opcode 2: before 507 < left 525, rollback = 2 [ 6799.834459] Lustre: 387155:0:(osd_handler.c:2081:osd_trans_dump_creds()) create: 3/12/6, destroy: 0/0/0 [ 6799.837483] Lustre: 387155:0:(osd_handler.c:2088:osd_trans_dump_creds()) attr_set: 3/3/0, xattr_set: 6/525/0 [ 6799.839532] Lustre: 387155:0:(osd_handler.c:2098:osd_trans_dump_creds()) write: 7/153/0, punch: 0/0/0, quota 1/3/0 [ 6799.841710] Lustre: 387155:0:(osd_handler.c:2105:osd_trans_dump_creds()) insert: 7/135/3, delete: 0/0/0 [ 6799.843578] Lustre: 387155:0:(osd_handler.c:2112:osd_trans_dump_creds()) ref_add: 1/1/0, ref_del: 0/0/0 [ 6805.471985] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 6805.473781] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6805.478117] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6805.480924] Lustre: Skipped 3 previous similar messages [ 6806.980077] LustreError: 387244:0:(obd_class.h:479:obd_check_dev()) Device 22 not setup [ 6806.981917] LustreError: 387244:0:(obd_class.h:479:obd_check_dev()) Skipped 11 previous similar messages [ 6807.012901] Lustre: server umount lustre-MDT0000 complete [ 6808.276169] LustreError: 386548:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1762173189 with bad export cookie 4952169246714611280 [ 6808.279755] LustreError: MGC192.168.202.147@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6808.279983] LustreError: 386548:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 6808.395566] Lustre: server umount lustre-MDT0001 complete [ 6819.931124] Lustre: server umount lustre-OST0000 complete [ 6829.502729] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing set_hostid [ 6832.736352] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing load_modules_local [ 6836.221898] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6838.673311] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6840.966751] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6843.246651] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6843.890147] Lustre: DEBUG MARKER: == conf-sanity test complete, duration 6658 sec ========== 07:33:44 (1762173224) [ 6844.463939] Lustre: DEBUG MARKER: === conf-sanity: start cleanup 07:33:45 (1762173225) === [ 6845.601170] Lustre: DEBUG MARKER: === conf-sanity: finish cleanup 07:33:46 (1762173226) === [ 6859.225197] Lustre: DEBUG MARKER: oleg247-server.virtnet: executing unload_modules_local [ 6860.332490] Key type lgssc unregistered [ 6860.465356] LNet: 392747:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 6860.469071] LNetError: 392747:0:(acceptor.c:264:lnet_acceptor_remove_socket()) Interface ens2 not found [ 6860.478352] LNet: Removed LNI 192.168.202.147@tcp [ 6860.798578] Key type .llcrypt unregistered [ 6860.800710] Key type ._llcrypt unregistered