[ 0.000000] Linux version 4.18.0rh8.10-debug (green@maintenance) (gcc version 8.5.0 20210514 (Red Hat 8.5.0-26) (GCC)) #2 SMP Mon Jul 14 01:24:22 EDT 2025 [ 0.000000] Command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers' [ 0.000000] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256 [ 0.000000] x86/fpu: Enabled xstate features 0x7, context size is 832 bytes, using 'standard' format. [ 0.000000] signal: max sigframe size: 1776 [ 0.000000] BIOS-provided physical RAM map: [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x00000000bffcdfff] usable [ 0.000000] BIOS-e820: [mem 0x00000000bffce000-0x00000000bfffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x0000000146dfffff] usable [ 0.000000] NX (Execute Disable) protection: active [ 0.000000] SMBIOS 2.8 present. [ 0.000000] DMI: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 0.000000] Hypervisor detected: KVM [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00 [ 0.000000] kvm-clock: using sched offset of 469149182 cycles [ 0.000000] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns [ 0.000000] tsc: Detected 2399.998 MHz processor [ 0.000000] e820: update [mem 0x00000000-0x00000fff] usable ==> reserved [ 0.000000] e820: remove [mem 0x000a0000-0x000fffff] usable [ 0.000000] last_pfn = 0x146e00 max_arch_pfn = 0x400000000 [ 0.000000] MTRR default type: write-back [ 0.000000] MTRR fixed ranges enabled: [ 0.000000] 00000-9FFFF write-back [ 0.000000] A0000-BFFFF uncachable [ 0.000000] C0000-FFFFF write-protect [ 0.000000] MTRR variable ranges enabled: [ 0.000000] 0 base 0000C0000000 mask 3FFFC0000000 uncachable [ 0.000000] 1 disabled [ 0.000000] 2 disabled [ 0.000000] 3 disabled [ 0.000000] 4 disabled [ 0.000000] 5 disabled [ 0.000000] 6 disabled [ 0.000000] 7 disabled [ 0.000000] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT [ 0.000000] last_pfn = 0xbffce max_arch_pfn = 0x400000000 [ 0.000000] found SMP MP-table at [mem 0x000f54b0-0x000f54bf] [ 0.000000] BRK [0xb3001000, 0xb3001fff] PGTABLE [ 0.000000] BRK [0xb3002000, 0xb3002fff] PGTABLE [ 0.000000] BRK [0xb3003000, 0xb3003fff] PGTABLE [ 0.000000] BRK [0xb3004000, 0xb3004fff] PGTABLE [ 0.000000] BRK [0xb3005000, 0xb3005fff] PGTABLE [ 0.000000] BRK [0xb3006000, 0xb3006fff] PGTABLE [ 0.000000] BRK [0xb3007000, 0xb3007fff] PGTABLE [ 0.000000] BRK [0xb3008000, 0xb3008fff] PGTABLE [ 0.000000] BRK [0xb3009000, 0xb3009fff] PGTABLE [ 0.000000] BRK [0xb300a000, 0xb300afff] PGTABLE [ 0.000000] BRK [0xb300b000, 0xb300bfff] PGTABLE [ 0.000000] BRK [0xb300c000, 0xb300cfff] PGTABLE [ 0.000000] RAMDISK: [mem 0xbcc54000-0xbffbffff] [ 0.000000] ACPI: Early table checksum verification disabled [ 0.000000] ACPI: RSDP 0x00000000000F52D0 000014 (v00 BOCHS ) [ 0.000000] ACPI: RSDT 0x00000000BFFE247C 000034 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACP 0x00000000BFFE2318 000074 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: DSDT 0x00000000BFFE0040 0022D8 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACS 0x00000000BFFE0000 000040 [ 0.000000] ACPI: APIC 0x00000000BFFE238C 000090 (v03 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: HPET 0x00000000BFFE241C 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: WAET 0x00000000BFFE2454 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: Reserving FACP table memory at [mem 0xbffe2318-0xbffe238b] [ 0.000000] ACPI: Reserving DSDT table memory at [mem 0xbffe0040-0xbffe2317] [ 0.000000] ACPI: Reserving FACS table memory at [mem 0xbffe0000-0xbffe003f] [ 0.000000] ACPI: Reserving APIC table memory at [mem 0xbffe238c-0xbffe241b] [ 0.000000] ACPI: Reserving HPET table memory at [mem 0xbffe241c-0xbffe2453] [ 0.000000] ACPI: Reserving WAET table memory at [mem 0xbffe2454-0xbffe247b] [ 0.000000] ACPI: Local APIC address 0xfee00000 [ 0.000000] No NUMA configuration found [ 0.000000] Faking a node at [mem 0x0000000000000000-0x0000000146dfffff] [ 0.000000] NODE_DATA(0) allocated [mem 0x1465a3000-0x1465cdfff] [ 0.000000] Reserving 256MB of memory at 2544MB for crashkernel (System RAM: 4205MB) [ 0.000000] Zone ranges: [ 0.000000] DMA [mem 0x0000000000001000-0x0000000000ffffff] [ 0.000000] DMA32 [mem 0x0000000001000000-0x00000000ffffffff] [ 0.000000] Normal [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Device empty [ 0.000000] Movable zone start for each node [ 0.000000] Early memory node ranges [ 0.000000] node 0: [mem 0x0000000000001000-0x000000000009efff] [ 0.000000] node 0: [mem 0x0000000000100000-0x00000000bffcdfff] [ 0.000000] node 0: [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Zeroed struct page in unavailable ranges: 4756 pages [ 0.000000] Initmem setup node 0 [mem 0x0000000000001000-0x0000000146dfffff] [ 0.000000] On node 0 totalpages: 1076588 [ 0.000000] DMA zone: 64 pages used for memmap [ 0.000000] DMA zone: 158 pages reserved [ 0.000000] DMA zone: 3998 pages, LIFO batch:0 [ 0.000000] DMA32 zone: 12224 pages used for memmap [ 0.000000] DMA32 zone: 782286 pages, LIFO batch:63 [ 0.000000] Normal zone: 4536 pages used for memmap [ 0.000000] Normal zone: 290304 pages, LIFO batch:63 [ 0.000000] ACPI: PM-Timer IO Port: 0x608 [ 0.000000] ACPI: Local APIC address 0xfee00000 [ 0.000000] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1]) [ 0.000000] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23 [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level) [ 0.000000] ACPI: IRQ0 used by override. [ 0.000000] ACPI: IRQ5 used by override. [ 0.000000] ACPI: IRQ9 used by override. [ 0.000000] ACPI: IRQ10 used by override. [ 0.000000] ACPI: IRQ11 used by override. [ 0.000000] Using ACPI (MADT) for SMP configuration information [ 0.000000] ACPI: HPET id: 0x8086a201 base: 0xfed00000 [ 0.000000] TSC deadline timer available [ 0.000000] smpboot: Allowing 4 CPUs, 0 hotplug CPUs [ 0.000000] kvm-guest: KVM setup pv remote TLB flush [ 0.000000] kvm-guest: setup PV sched yield [ 0.000000] PM: Registered nosave memory: [mem 0x00000000-0x00000fff] [ 0.000000] PM: Registered nosave memory: [mem 0x0009f000-0x0009ffff] [ 0.000000] PM: Registered nosave memory: [mem 0x000a0000-0x000effff] [ 0.000000] PM: Registered nosave memory: [mem 0x000f0000-0x000fffff] [ 0.000000] PM: Registered nosave memory: [mem 0xbffce000-0xbfffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xc0000000-0xfeffbfff] [ 0.000000] PM: Registered nosave memory: [mem 0xfeffc000-0xfeffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xff000000-0xfffbffff] [ 0.000000] PM: Registered nosave memory: [mem 0xfffc0000-0xffffffff] [ 0.000000] [mem 0xc0000000-0xfeffbfff] available for PCI devices [ 0.000000] Booting paravirtualized kernel on KVM [ 0.000000] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns [ 0.000000] setup_percpu: NR_CPUS:8192 nr_cpumask_bits:4 nr_cpu_ids:4 nr_node_ids:1 [ 0.000000] percpu: Embedded 63 pages/cpu s221184 r8192 d28672 u524288 [ 0.000000] pcpu-alloc: s221184 r8192 d28672 u524288 alloc=1*2097152 [ 0.000000] pcpu-alloc: [0] 0 1 2 3 [ 0.000000] kvm-guest: PV spinlocks enabled [ 0.000000] PV qspinlock hash table entries: 256 (order: 0, 4096 bytes, linear) [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 1059606 [ 0.000000] Policy zone: Normal [ 0.000000] Kernel command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] Specific versions of hardware are certified with Red Hat Enterprise Linux 8. Please see the list of hardware certified with Red Hat Enterprise Linux 8 at https://catalog.redhat.com. [ 0.000000] audit: disabled (until reboot) [ 0.000000] software IO TLB: area num 4. [ 0.000000] Memory: 2829652K/4306352K available (18435K kernel code, 11221K rwdata, 7248K rodata, 2908K init, 18040K bss, 524580K reserved, 0K cma-reserved) [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1 [ 0.000000] kmemleak: Kernel memory leak detector disabled [ 0.000000] ftrace: allocating 41240 entries in 162 pages [ 0.000000] ftrace: allocated 162 pages with 3 groups [ 0.000000] rcu: Hierarchical RCU implementation. [ 0.000000] rcu: RCU event tracing is enabled. [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=8192 to nr_cpu_ids=4. [ 0.000000] rcu: RCU callback double-/use-after-free debug enabled. [ 0.000000] Rude variant of Tasks RCU enabled. [ 0.000000] Tracing variant of Tasks RCU enabled. [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies. [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4 [ 0.000000] NR_IRQS: 524544, nr_irqs: 456, preallocated irqs: 16 [ 0.000000] random: get_random_bytes called from start_kernel+0x622/0x9a8 with crng_init=0 [ 0.001000] Console: colour *CGA 80x25 [ 0.001000] printk: console [ttyS1] enabled [ 0.001000] ACPI: Core revision 20220331 [ 0.001000] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns [ 0.001000] hpet clockevent registered [ 0.001012] APIC: Switch to symmetric I/O mode setup [ 0.003238] x2apic enabled [ 0.004010] Switched APIC routing to physical x2apic. [ 0.005017] kvm-guest: setup PV IPIs [ 0.007845] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1 [ 0.008000] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 0.008025] Calibrating delay loop (skipped) preset value.. 4799.99 BogoMIPS (lpj=2399998) [ 0.010008] pid_max: default: 32768 minimum: 301 [ 0.012068] LSM: Security Framework initializing [ 0.013062] Yama: becoming mindful. [ 0.014042] SELinux: Initializing. [ 0.015073] *** VALIDATE selinux *** [ 0.024026] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, vmalloc) [ 0.029700] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, vmalloc) [ 0.030144] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.031092] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.032149] *** VALIDATE tmpfs *** [ 0.034234] *** VALIDATE proc *** [ 0.035260] *** VALIDATE cgroup *** [ 0.036012] *** VALIDATE cgroup2 *** [ 0.037296] x86/cpu: User Mode Instruction Prevention (UMIP) activated [ 0.038166] Last level iTLB entries: 4KB 0, 2MB 0, 4MB 0 [ 0.039014] Last level dTLB entries: 4KB 0, 2MB 0, 4MB 0, 1GB 0 [ 0.040034] Spectre V2 : User space: Vulnerable [ 0.042008] Speculative Store Bypass: Vulnerable [ 0.045400] debug: unmapping init [mem 0xffffffff96659000-0xffffffff96660fff] [ 0.047971] smpboot: CPU0: Intel(R) Xeon(R) CPU E5-2695 v2 @ 2.40GHz (family: 0x6, model: 0x3e, stepping: 0x4) [ 0.048742] Performance Events: IvyBridge events, full-width counters, Intel PMU driver. [ 0.049029] ... version: 2 [ 0.050015] ... bit width: 48 [ 0.051015] ... generic registers: 4 [ 0.052019] ... value mask: 0000ffffffffffff [ 0.053023] ... max period: 00007fffffffffff [ 0.054014] ... fixed-purpose events: 3 [ 0.055014] ... event mask: 000000070000000f [ 0.056312] rcu: Hierarchical SRCU implementation. [ 0.058504] smp: Bringing up secondary CPUs ... [ 0.059580] x86: Booting SMP configuration: [ 0.060032] .... node #0, CPUs: #1 #2 #3 [ 0.063568] smp: Brought up 1 node, 4 CPUs [ 0.065014] smpboot: Max logical packages: 1 [ 0.066022] smpboot: Total of 4 processors activated (19199.98 BogoMIPS) [ 0.145753] node 0 deferred pages initialised in 76ms [ 0.149213] devtmpfs: initialized [ 0.150277] x86/mm: Memory block size: 128MB [ 0.152894] gcov: version magic: 0x41383552 [ 0.154308] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns [ 0.155116] futex hash table entries: 1024 (order: 4, 65536 bytes, vmalloc) [ 0.156277] pinctrl core: initialized pinctrl subsystem [ 0.157761] ************************************************************* [ 0.158015] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.159015] ** ** [ 0.160014] ** IOMMU DebugFS SUPPORT HAS BEEN ENABLED IN THIS KERNEL ** [ 0.161014] ** ** [ 0.162017] ** This means that this kernel is built to expose internal ** [ 0.163015] ** IOMMU data structures, which may compromise security on ** [ 0.164017] ** your system. ** [ 0.165018] ** ** [ 0.166016] ** If you see this message and you are not debugging the ** [ 0.167017] ** kernel, report this immediately to your vendor! ** [ 0.168014] ** ** [ 0.169015] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.170016] ************************************************************* [ 0.171725] NET: Registered protocol family 16 [ 0.172470] DMA: preallocated 512 KiB GFP_KERNEL pool for atomic allocations [ 0.173072] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations [ 0.174061] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations [ 0.176042] cpuidle: using governor menu [ 0.178183] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5 [ 0.180541] PCI: Using configuration type 1 for base access [ 0.183130] core: PMU erratum BJ122, BV98, HSD29 worked around, HT is on [ 0.188197] HugeTLB registered 1.00 GiB page size, pre-allocated 0 pages [ 0.189026] HugeTLB registered 2.00 MiB page size, pre-allocated 0 pages [ 0.191047] cryptd: max_cpu_qlen set to 1000 [ 0.192413] ACPI: Added _OSI(Module Device) [ 0.193017] ACPI: Added _OSI(Processor Device) [ 0.194016] ACPI: Added _OSI(3.0 _SCP Extensions) [ 0.195017] ACPI: Added _OSI(Processor Aggregator Device) [ 0.199368] ACPI: 1 ACPI AML tables successfully acquired and loaded [ 0.201689] ACPI: Interpreter enabled [ 0.202070] ACPI: PM: (supports S0 S3 S4 S5) [ 0.203015] ACPI: Using IOAPIC for interrupt routing [ 0.204144] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug [ 0.205454] ACPI: Enabled 2 GPEs in block 00 to 0F [ 0.215128] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff]) [ 0.216048] acpi PNP0A03:00: _OSC: OS supports [ASPM ClockPM Segments MSI HPX-Type3] [ 0.217044] acpi PNP0A03:00: _OSC: not requesting OS control; OS requires [ExtendedConfig ASPM ClockPM MSI] [ 0.218153] acpi PNP0A03:00: fail to add MMCONFIG information, can't access extended PCI configuration space under this bridge. [ 0.220463] acpiphp: Slot [2] registered [ 0.221138] acpiphp: Slot [5] registered [ 0.222116] acpiphp: Slot [6] registered [ 0.223101] acpiphp: Slot [7] registered [ 0.224107] acpiphp: Slot [8] registered [ 0.225094] acpiphp: Slot [9] registered [ 0.226146] acpiphp: Slot [10] registered [ 0.227118] acpiphp: Slot [3] registered [ 0.228090] acpiphp: Slot [4] registered [ 0.229092] acpiphp: Slot [11] registered [ 0.230153] acpiphp: Slot [12] registered [ 0.231105] acpiphp: Slot [13] registered [ 0.232094] acpiphp: Slot [14] registered [ 0.233140] acpiphp: Slot [15] registered [ 0.234127] acpiphp: Slot [16] registered [ 0.235092] acpiphp: Slot [17] registered [ 0.236093] acpiphp: Slot [18] registered [ 0.237102] acpiphp: Slot [19] registered [ 0.238096] acpiphp: Slot [20] registered [ 0.239079] acpiphp: Slot [21] registered [ 0.240143] acpiphp: Slot [22] registered [ 0.241093] acpiphp: Slot [23] registered [ 0.242106] acpiphp: Slot [24] registered [ 0.243097] acpiphp: Slot [25] registered [ 0.244128] acpiphp: Slot [26] registered [ 0.245106] acpiphp: Slot [27] registered [ 0.246089] acpiphp: Slot [28] registered [ 0.247103] acpiphp: Slot [29] registered [ 0.248090] acpiphp: Slot [30] registered [ 0.249088] acpiphp: Slot [31] registered [ 0.250060] PCI host bridge to bus 0000:00 [ 0.251024] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window] [ 0.252021] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window] [ 0.253022] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window] [ 0.254023] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window] [ 0.255016] pci_bus 0000:00: root bus resource [mem 0xe0000000000-0xe007fffffff window] [ 0.256022] pci_bus 0000:00: root bus resource [bus 00-ff] [ 0.257254] pci 0000:00:00.0: [8086:1237] type 00 class 0x060000 [ 0.259000] pci 0000:00:01.0: [8086:7000] type 00 class 0x060100 [ 0.261294] pci 0000:00:01.1: [8086:7010] type 00 class 0x010180 [ 0.267017] pci 0000:00:01.1: reg 0x20: [io 0xc320-0xc32f] [ 0.270053] pci 0000:00:01.1: legacy IDE quirk: reg 0x10: [io 0x01f0-0x01f7] [ 0.271018] pci 0000:00:01.1: legacy IDE quirk: reg 0x14: [io 0x03f6] [ 0.272014] pci 0000:00:01.1: legacy IDE quirk: reg 0x18: [io 0x0170-0x0177] [ 0.273014] pci 0000:00:01.1: legacy IDE quirk: reg 0x1c: [io 0x0376] [ 0.274570] pci 0000:00:01.3: [8086:7113] type 00 class 0x068000 [ 0.275687] pci 0000:00:01.3: quirk: [io 0x0600-0x063f] claimed by PIIX4 ACPI [ 0.276043] pci 0000:00:01.3: quirk: [io 0x0700-0x070f] claimed by PIIX4 SMB [ 0.277745] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 [ 0.280015] pci 0000:00:02.0: reg 0x10: [io 0xc300-0xc31f] [ 0.288865] pci 0000:00:02.0: reg 0x20: [mem 0xe0000000000-0xe0000003fff 64bit pref] [ 0.290020] pci 0000:00:02.0: reg 0x30: [mem 0xfeb80000-0xfebbffff pref] [ 0.293570] pci 0000:00:05.0: [1af4:1001] type 00 class 0x010000 [ 0.295907] pci 0000:00:05.0: reg 0x10: [io 0xc000-0xc07f] [ 0.298019] pci 0000:00:05.0: reg 0x14: [mem 0xfebc0000-0xfebc0fff] [ 0.305055] pci 0000:00:05.0: reg 0x20: [mem 0xe0000004000-0xe0000007fff 64bit pref] [ 0.311740] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 [ 0.314037] pci 0000:00:06.0: reg 0x10: [io 0xc080-0xc0ff] [ 0.317018] pci 0000:00:06.0: reg 0x14: [mem 0xfebc1000-0xfebc1fff] [ 0.325023] pci 0000:00:06.0: reg 0x20: [mem 0xe0000008000-0xe000000bfff 64bit pref] [ 0.330882] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 [ 0.333012] pci 0000:00:07.0: reg 0x10: [io 0xc100-0xc17f] [ 0.336018] pci 0000:00:07.0: reg 0x14: [mem 0xfebc2000-0xfebc2fff] [ 0.341018] pci 0000:00:07.0: reg 0x20: [mem 0xe000000c000-0xe000000ffff 64bit pref] [ 0.347125] pci 0000:00:08.0: [1af4:1001] type 00 class 0x010000 [ 0.349991] pci 0000:00:08.0: reg 0x10: [io 0xc180-0xc1ff] [ 0.351894] pci 0000:00:08.0: reg 0x14: [mem 0xfebc3000-0xfebc3fff] [ 0.357016] pci 0000:00:08.0: reg 0x20: [mem 0xe0000010000-0xe0000013fff 64bit pref] [ 0.363978] pci 0000:00:09.0: [1af4:1001] type 00 class 0x010000 [ 0.369013] pci 0000:00:09.0: reg 0x10: [io 0xc200-0xc27f] [ 0.376023] pci 0000:00:09.0: reg 0x14: [mem 0xfebc4000-0xfebc4fff] [ 0.391041] pci 0000:00:09.0: reg 0x20: [mem 0xe0000014000-0xe0000017fff 64bit pref] [ 0.403928] pci 0000:00:0a.0: [1af4:1001] type 00 class 0x010000 [ 0.413019] pci 0000:00:0a.0: reg 0x10: [io 0xc280-0xc2ff] [ 0.419020] pci 0000:00:0a.0: reg 0x14: [mem 0xfebc5000-0xfebc5fff] [ 0.435016] pci 0000:00:0a.0: reg 0x20: [mem 0xe0000018000-0xe000001bfff 64bit pref] [ 0.447644] ACPI: PCI: Interrupt link LNKA configured for IRQ 10 [ 0.449305] ACPI: PCI: Interrupt link LNKB configured for IRQ 10 [ 0.451354] ACPI: PCI: Interrupt link LNKC configured for IRQ 11 [ 0.453319] ACPI: PCI: Interrupt link LNKD configured for IRQ 11 [ 0.457223] ACPI: PCI: Interrupt link LNKS configured for IRQ 9 [ 0.461113] iommu: Default domain type: Passthrough [ 0.462394] SCSI subsystem initialized [ 0.464349] ACPI: bus type USB registered [ 0.465223] usbcore: registered new interface driver usbfs [ 0.467096] usbcore: registered new interface driver hub [ 0.468100] usbcore: registered new device driver usb [ 0.470133] pps_core: LinuxPPS API ver. 1 registered [ 0.471008] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti [ 0.473069] PTP clock support registered [ 0.475072] EDAC MC: Ver: 3.0.0 [ 0.476423] PCI: Using ACPI for IRQ routing [ 0.477013] PCI: pci_cache_line_size set to 64 bytes [ 0.477255] e820: reserve RAM buffer [mem 0x0009fc00-0x0009ffff] [ 0.477265] e820: reserve RAM buffer [mem 0xbffce000-0xbfffffff] [ 0.477269] e820: reserve RAM buffer [mem 0x146e00000-0x147ffffff] [ 0.477768] NetLabel: Initializing [ 0.479009] NetLabel: domain hash size = 128 [ 0.479941] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO [ 0.482124] NetLabel: unlabeled traffic allowed by default [ 0.483277] vgaarb: loaded [ 0.484234] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0 [ 0.485007] hpet0: 3 comparators, 64-bit 100.000000 MHz counter [ 0.489000] clocksource: Switched to clocksource kvm-clock [ 0.593762] VFS: Disk quotas dquot_6.6.0 [ 0.595229] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) [ 0.597682] *** VALIDATE ramfs *** [ 0.598798] *** VALIDATE hugetlbfs *** [ 0.600196] pnp: PnP ACPI init [ 0.601376] pnp 00:00: Plug and Play ACPI device, IDs PNP0303 (active) [ 0.601446] pnp 00:01: Plug and Play ACPI device, IDs PNP0f13 (active) [ 0.601473] pnp 00:02: [dma 2] [ 0.601541] pnp 00:02: Plug and Play ACPI device, IDs PNP0700 (active) [ 0.601609] pnp 00:03: Plug and Play ACPI device, IDs PNP0501 (active) [ 0.601685] pnp 00:04: Plug and Play ACPI device, IDs PNP0501 (active) [ 0.601747] pnp 00:05: Plug and Play ACPI device, IDs PNP0b00 (active) [ 0.602339] pnp: PnP ACPI: found 6 devices [ 0.620371] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns [ 0.623365] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window] [ 0.625156] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window] [ 0.626944] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window] [ 0.629296] pci_bus 0000:00: resource 7 [mem 0xc0000000-0xfebfffff window] [ 0.631046] pci_bus 0000:00: resource 8 [mem 0xe0000000000-0xe007fffffff window] [ 0.632698] NET: Registered protocol family 2 [ 0.634769] IP idents hash table entries: 131072 (order: 8, 1048576 bytes, vmalloc) [ 0.638948] tcp_listen_portaddr_hash hash table entries: 4096 (order: 5, 163840 bytes, vmalloc) [ 0.642502] TCP established hash table entries: 65536 (order: 7, 524288 bytes, vmalloc) [ 0.647266] TCP bind hash table entries: 65536 (order: 9, 2097152 bytes, vmalloc) [ 0.650762] TCP: Hash tables configured (established 65536 bind 65536) [ 0.653784] MPTCP token hash table entries: 8192 (order: 6, 393216 bytes, vmalloc) [ 0.656303] UDP hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 0.658921] UDP-Lite hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 0.661839] NET: Registered protocol family 1 [ 0.664320] RPC: Registered named UNIX socket transport module. [ 0.666189] RPC: Registered udp transport module. [ 0.667542] RPC: Registered tcp transport module. [ 0.669310] RPC: Registered tcp NFSv4.1 backchannel transport module. [ 0.671550] NET: Registered protocol family 44 [ 0.673340] pci 0000:00:00.0: Limiting direct PCI/PCI transfers [ 0.677793] pci 0000:00:01.0: PIIX3: Enabling Passive Release [ 0.679605] pci 0000:00:01.0: Activating ISA DMA hang workarounds [ 0.681898] PCI: CLS 0 bytes, default 64 [ 0.683551] Unpacking initramfs... [ 2.003970] debug: unmapping init [mem 0xffff92687cc54000-0xffff92687ffbffff] [ 2.007811] PCI-DMA: Using software bounce buffering for IO (SWIOTLB) [ 2.009837] software IO TLB: mapped [mem 0x00000000b8c54000-0x00000000bcc54000] (64MB) [ 2.012375] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 2.449947] Initialise system trusted keyrings [ 2.451516] Key type blacklist registered [ 2.453610] workingset: timestamp_bits=36 max_order=20 bucket_order=0 [ 2.462233] zbud: loaded [ 2.464628] *** VALIDATE nfs *** [ 2.465636] *** VALIDATE nfs4 *** [ 2.467351] pstore: using deflate compression [ 2.470304] Platform Keyring initialized [ 2.549898] NET: Registered protocol family 38 [ 2.551528] Key type asymmetric registered [ 2.552757] Asymmetric key parser 'x509' registered [ 2.554419] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 247) [ 2.557321] io scheduler mq-deadline registered [ 2.559965] io scheduler kyber registered [ 2.561642] io scheduler bfq registered [ 2.562920] atomic64_test: passed for x86-64 platform with CX8 and with SSE [ 2.565736] shpchp: Standard Hot Plug PCI Controller Driver version: 0.4 [ 2.568465] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0 [ 2.571164] ACPI: Power Button [PWRF] [ 2.576519] ACPI: \_SB_.LNKB: Enabled at IRQ 10 [ 2.581464] ACPI: \_SB_.LNKA: Enabled at IRQ 11 [ 2.592445] ACPI: \_SB_.LNKC: Enabled at IRQ 11 [ 2.599231] ACPI: \_SB_.LNKD: Enabled at IRQ 10 [ 2.612539] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled [ 2.642055] 00:03: ttyS1 at I/O 0x2f8 (irq = 3, base_baud = 115200) is a 16550A [ 2.672376] 00:04: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A [ 2.676269] Non-volatile memory driver v1.3 [ 2.677265] Linux agpgart interface v0.103 [ 2.705328] virtio_blk virtio1: [vda] 149784 512-byte logical blocks (76.7 MB/73.1 MiB) [ 2.707157] vda: detected capacity change from 0 to 76689408 [ 2.718343] virtio_blk virtio2: [vdb] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) [ 2.721322] vdb: detected capacity change from 0 to 1073741824 [ 2.737412] virtio_blk virtio3: [vdc] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 2.740471] vdc: detected capacity change from 0 to 2621440000 [ 2.754419] virtio_blk virtio4: [vdd] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 2.756377] vdd: detected capacity change from 0 to 2621440000 [ 2.767363] virtio_blk virtio5: [vde] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 2.769658] vde: detected capacity change from 0 to 4294967296 [ 2.784460] virtio_blk virtio6: [vdf] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 2.786757] vdf: detected capacity change from 0 to 4294967296 [ 2.791507] libphy: Fixed MDIO Bus: probed [ 2.800061] usbcore: registered new interface driver usbserial_generic [ 2.802909] usbserial: USB Serial support registered for generic [ 2.805240] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12 [ 2.809567] serio: i8042 KBD port at 0x60,0x64 irq 1 [ 2.811328] serio: i8042 AUX port at 0x60,0x64 irq 12 [ 2.813721] mousedev: PS/2 mouse device common for all mice [ 2.816878] rtc_cmos 00:05: RTC can wake from S4 [ 2.819551] rtc_cmos 00:05: registered as rtc0 [ 2.819692] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input1 [ 2.821087] rtc_cmos 00:05: alarms up to one day, y3k, 242 bytes nvram, hpet irqs [ 2.826264] intel_pstate: CPU model not supported [ 2.828631] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input4 [ 2.831250] hid: raw HID events driver (C) Jiri Kosina [ 2.832980] usbcore: registered new interface driver usbhid [ 2.834370] usbhid: USB HID core driver [ 2.835690] drop_monitor: Initializing network drop monitor service [ 2.835692] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input3 [ 2.836952] Initializing XFRM netlink socket [ 2.840620] NET: Registered protocol family 10 [ 2.842825] Segment Routing with IPv6 [ 2.843837] NET: Registered protocol family 17 [ 2.845365] mpls_gso: MPLS GSO support [ 2.846155] start plist test [ 2.847740] end plist test [ 2.848964] RAS: Correctable Errors collector initialized. [ 2.850437] AVX version of gcm_enc/dec engaged. [ 2.851279] AES CTR mode by8 optimization enabled [ 2.909900] sched_clock: Marking stable (2909881618, 0)->(3991287902, -1081406284) [ 2.912121] registered taskstats version 1 [ 2.913923] Loading compiled-in X.509 certificates [ 2.915213] zswap: loaded using pool lzo/zbud [ 2.937598] Key type big_key registered [ 2.948361] Key type encrypted registered [ 2.949913] ima: No TPM chip found, activating TPM-bypass! [ 2.951944] ima: Allocated hash algorithm: sha1 [ 2.953148] ima: No architecture policies found [ 2.954646] evm: Initialising EVM extended attributes: [ 2.956072] evm: security.selinux [ 2.956900] evm: security.ima [ 2.957793] evm: security.capability [ 2.958761] evm: HMAC attrs: 0x1 [ 2.960509] rtc_cmos 00:05: setting system clock to 2026-09-07 16:40:36 UTC (1788799236) [ 2.965541] debug: unmapping init [mem 0xffffffff97603000-0xffffffff977fffff] [ 2.968048] debug: unmapping init [mem 0xffffffff96382000-0xffffffff96658fff] [ 2.976080] Write protecting the kernel read-only data: 28672k [ 2.978796] debug: unmapping init [mem 0xffffffff94a03000-0xffffffff94bfffff] [ 2.981219] debug: unmapping init [mem 0xffffffff95314000-0xffffffff953fffff] [ 3.011477] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 3.019588] systemd[1]: Detected virtualization kvm. [ 3.021249] systemd[1]: Detected architecture x86-64. [ 3.022828] systemd[1]: Running in initial RAM disk. [ 3.045768] systemd[1]: No hostname configured. [ 3.046703] systemd[1]: Set hostname to . [ 3.048288] random: systemd: uninitialized urandom read (16 bytes read) [ 3.050158] systemd[1]: Initializing machine ID from random generator. [ 3.090626] random: ln: uninitialized urandom read (6 bytes read) [ 3.173925] random: systemd: uninitialized urandom read (16 bytes read) [ 3.177950] systemd[1]: Reached target Swap. [ 3.187358] systemd[1]: Reached target Initrd Root Device. [ 3.192718] systemd[1]: Listening on Journal Socket. [ 3.774279] device-mapper: uevent: version 1.0.3 [ 3.776180] device-mapper: ioctl: 4.46.0-ioctl (2022-02-22) initialised: dm-devel@redhat.com [ 4.414580] random: fast init done [ 4.462400] virtio_net virtio0 ens2: renamed from eth0 [ 4.479512] libata version 3.00 loaded. [ 4.482892] ata_piix 0000:00:01.1: version 2.13 [ 4.496364] scsi host0: ata_piix [ 4.564877] scsi host1: ata_piix [ 4.566607] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc320 irq 14 [ 4.569206] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc328 irq 15 [ 9.463347] random: crng init done [ 9.464701] random: 7 urandom warning(s) missed due to ratelimiting [ 9.885222] EXT4-fs (nbd0): mounted filesystem with ordered data mode. Opts: (null) [ 11.058225] printk: systemd: 26 output lines suppressed due to ratelimiting [ 11.337780] SELinux: Disabled at runtime. [ 11.403463] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 11.412467] systemd[1]: Detected virtualization kvm. [ 11.414345] systemd[1]: Detected architecture x86-64. [ 11.966787] systemd[1]: initrd-switch-root.service: Succeeded. [ 11.969628] systemd[1]: Stopped Switch Root. [ 11.974247] systemd[1]: systemd-journald.service: Service has no hold-off time (RestartSec=0), scheduling restart. [ 11.977387] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. [ 11.979603] systemd[1]: Stopped Journal Service. [ 11.991828] systemd[1]: Starting Journal Service... [ 11.999612] systemd[1]: Created slice system-getty.slice. [ 12.190485] Adding 1048572k swap on /dev/vdb. Priority:-2 extents:1 across:1048572k FS [ 12.557851] squashfs: version 4.0 (2009/01/31) Phillip Lougher [ 12.942231] input: PC Speaker as /devices/platform/pcspkr/input/input5 [ 12.998784] piix4_smbus 0000:00:01.3: SMBus Host Controller at 0x700, revision 0 [ 13.153649] RAPL PMU: API unit is 2^-32 Joules, 0 fixed counters, 10737418240 ms ovfl timer [ 13.164873] EDAC sbridge: Seeking for: PCI ID 8086:0ea0 [ 13.164909] EDAC sbridge: Ver: 1.1.2 [ 14.894427] Key type dns_resolver registered [ 15.202843] NFS: Registering the id_resolver key type [ 15.204749] Key type id_resolver registered [ 15.206339] Key type id_legacy registered [ 50.665946] libcfs: loading out-of-tree module taints kernel. [ 50.696330] Key type ._llcrypt registered [ 50.697868] Key type .llcrypt registered [ 50.853678] Lustre: DEBUG MARKER: oleg221-server.virtnet: executing set_hostid [ 73.366692] Lustre: DEBUG MARKER: oleg221-server.virtnet: executing load_modules_local [ 75.494301] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 75.519835] alg: No test for adler32 (adler32-zlib) [ 77.301639] Lustre: Lustre: Build Version: 2.17.58_2_gefa5002 [ 78.598780] LNet: Added LNI 192.168.202.121@tcp [8/256/0/180] [ 80.655720] Key type lgssc registered [ 82.909143] Lustre: Echo OBD driver; http://www.lustre.org/ [ 104.247705] ZFS: Loaded module v2.3.2-1, ZFS pool version 5000, ZFS filesystem version 5 [ 136.687079] hrtimer: interrupt took 2181467 ns [ 154.078689] Lustre: DEBUG MARKER: oleg221-server.virtnet: executing load_modules_local [ 170.692182] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 170.746182] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 172.044754] Lustre: First target registration detected: opening a one-time 24-hour registration window. Use 'lctl set_param allow_register=0' to close it early, or 'lctl set_param -P allow_register=0' to lock down permanently. [ 172.097602] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 172.170351] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 172.330642] Lustre: lustre-MDT0000: new disk, initializing [ 172.448672] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 172.529491] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 177.098514] Lustre: DEBUG MARKER: oleg221-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 195.523259] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 195.628279] Lustre: 6513:0:(mgs_llog.c:1462:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 195.682268] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 195.693164] Lustre: Skipped 1 previous similar message [ 195.791562] Lustre: lustre-MDT0001: new disk, initializing [ 195.885470] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 195.937478] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 195.948045] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 203.020940] Lustre: DEBUG MARKER: oleg221-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 209.318240] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 221.042984] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 221.267442] Lustre: lustre-OST0000: new disk, initializing [ 221.271420] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 221.276171] Lustre: 8450:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 221.339881] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 221.794235] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 221.811209] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 221.945817] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 229.385989] Lustre: DEBUG MARKER: oleg221-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 248.058563] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 248.257770] Lustre: lustre-OST0001: new disk, initializing [ 248.261727] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 248.268826] Lustre: 9523:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 248.346750] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 254.537079] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 254.565231] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 254.749329] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 256.466568] Lustre: DEBUG MARKER: oleg221-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 270.801686] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 283.348530] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 290.584702] Lustre: DEBUG MARKER: oleg221-server.virtnet: executing check_logdir /tmp/testlogs/ [ 296.670790] Lustre: DEBUG MARKER: oleg221-server.virtnet: executing yml_node [ 301.352062] Lustre: DEBUG MARKER: Client: 2.17.58.2 [ 305.288124] Lustre: DEBUG MARKER: MDS: 2.17.58.2 [ 307.755995] Lustre: DEBUG MARKER: OSS: 2.17.58.2 [ 310.250239] Lustre: DEBUG MARKER: -----============= acceptance-small: conf-sanity ============----- Mon Sep 7 12:45:41 EDT 2026 [ 334.886699] Lustre: DEBUG MARKER: excepting tests: 55 63 67 32newtarball [ 336.837578] Lustre: DEBUG MARKER: skipping tests SLOW=no: 45 69 106 111 114 [ 354.790110] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 354.820530] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 354.851873] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 357.359913] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 357.365365] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 357.376144] Lustre: Skipped 2 previous similar messages [ 357.391100] Lustre: Skipped 3 previous similar messages [ 361.013401] Lustre: server umount lustre-MDT0000 complete [ 366.407283] LustreError: 6505:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788799599 with bad export cookie 12127145325200087512 [ 366.414681] LustreError: MGC192.168.202.121@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 366.426666] LustreError: 6505:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 367.589153] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 367.592468] LustreError: 6518:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 367.604415] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 367.636089] Lustre: Skipped 1 previous similar message [ 367.663726] LustreError: 6518:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 6 previous similar messages [ 371.618540] LustreError: 6520:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 371.620782] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 371.624467] LustreError: 6520:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 371.626593] Lustre: Skipped 2 previous similar messages [ 373.090247] Lustre: server umount lustre-MDT0001 complete [ 377.984365] Lustre: server umount lustre-OST0000 complete [ 384.159402] Lustre: server umount lustre-OST0001 complete [ 398.166994] Lustre: DEBUG MARKER: oleg221-server.virtnet: executing set_hostid [ 409.196478] Lustre: DEBUG MARKER: oleg221-server.virtnet: executing load_modules_local [ 449.464557] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 449.712135] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 449.744214] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 449.830899] Lustre: lustre-MDT0000: new disk, initializing [ 449.897332] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 449.910810] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 454.901773] Lustre: DEBUG MARKER: oleg221-server.virtnet: executing set_default_debug -1 all [ 466.210102] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 466.379193] Lustre: 15080:0:(mgs_llog.c:1462:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 466.452592] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 466.460036] Lustre: Skipped 1 previous similar message [ 466.520508] Lustre: lustre-MDT0001: new disk, initializing [ 466.682504] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 466.799711] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 466.817855] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 471.288461] Lustre: DEBUG MARKER: oleg221-server.virtnet: executing set_default_debug -1 all [ 482.353546] Lustre: DEBUG MARKER: oleg221-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 489.512683] Lustre: DEBUG MARKER: oleg221-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 496.808425] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 497.125525] Lustre: lustre-OST0000: new disk, initializing [ 497.130363] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 497.136816] Lustre: 17048:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 497.247691] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 498.938474] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 498.954527] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 499.066806] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 503.679497] Lustre: DEBUG MARKER: oleg221-server.virtnet: executing set_default_debug -1 all [ 514.186480] Lustre: DEBUG MARKER: oleg221-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 519.018142] Lustre: DEBUG MARKER: oleg221-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 519.217298] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 524.369329] Lustre: DEBUG MARKER: oleg221-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 524.629931] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 529.894350] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 529.903333] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 529.904641] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 529.908185] Lustre: Skipped 1 previous similar message [ 529.916372] LustreError: Skipped 1 previous similar message [ 533.098459] Lustre: server umount lustre-OST0000 complete [ 538.594392] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 538.599286] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 538.606499] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 538.610172] Lustre: Skipped 1 previous similar message [ 544.111478] Lustre: server umount lustre-MDT0000 complete [ 545.255226] LustreError: 15086:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 545.281435] LustreError: 15086:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 548.598207] LustreError: 15072:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788799782 with bad export cookie 12127145325200089045 [ 548.606837] LustreError: MGC192.168.202.121@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 548.613335] LustreError: 15072:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 548.919617] Lustre: server umount lustre-MDT0001 complete [ 553.394720] Lustre: DEBUG MARKER: == conf-sanity test 33a: Mount ost with a large index number ========================================================== 12:49:45 (1788799785) [ 556.746156] Lustre: DEBUG MARKER: SKIP: conf-sanity test_33a mixed loopback and real device not working [ 559.858713] Lustre: DEBUG MARKER: == conf-sanity test 33b: Drop cancel during umount ======= 12:49:51 (1788799791) [ 569.227217] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 569.845258] LustreError: 19458:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 569.909972] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 574.960235] LustreError: 19459:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 575.354066] Lustre: DEBUG MARKER: oleg221-server.virtnet: executing set_default_debug -1 all [ 585.187432] LustreError: 19459:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 585.205370] LustreError: 19459:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 585.534130] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 590.947580] Lustre: DEBUG MARKER: oleg221-server.virtnet: executing set_default_debug -1 all [ 600.292482] Lustre: DEBUG MARKER: oleg221-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 607.536336] Lustre: DEBUG MARKER: oleg221-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 615.672249] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 623.455672] Lustre: DEBUG MARKER: oleg221-server.virtnet: executing set_default_debug -1 all [ 632.100028] Lustre: DEBUG MARKER: oleg221-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 655.331612] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 655.337699] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 655.344329] Lustre: Skipped 1 previous similar message [ 655.347971] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 655.352882] Lustre: Skipped 3 previous similar messages [ 659.061950] Lustre: server umount lustre-OST0000 complete [ 670.020331] Lustre: server umount lustre-MDT0000 complete [ 670.688689] LustreError: 19459:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 670.709660] LustreError: 19459:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 674.574437] LustreError: 19439:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788799908 with bad export cookie 12127145325200089927 [ 674.578977] LustreError: MGC192.168.202.121@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 674.606733] LustreError: 19439:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 675.263803] Lustre: server umount lustre-MDT0001 complete [ 689.504705] Lustre: DEBUG MARKER: oleg221-server.virtnet: executing unload_modules_local [ 693.353931] Key type lgssc unregistered [ 693.811679] LNet: 23012:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 693.834197] LNetError: 23012:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 693.858019] LNet: Removed LNI 192.168.202.121@tcp [ 694.719190] Key type .llcrypt unregistered [ 694.721192] Key type ._llcrypt unregistered [ 723.892589] Key type ._llcrypt registered [ 723.900195] Key type .llcrypt registered [ 724.030216] Lustre: DEBUG MARKER: oleg221-server.virtnet: executing load_modules_local [ 725.391793] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 725.426027] alg: No test for adler32 (adler32-zlib) [ 726.518708] Lustre: Lustre: Build Version: 2.17.58_2_gefa5002 [ 726.825432] LNet: Added LNI 192.168.202.121@tcp [8/256/0/180] [ 728.543229] Key type lgssc registered [ 730.073216] Lustre: Echo OBD driver; http://www.lustre.org/ [ 739.924797] loop: module loaded [ 740.035527] loop0: detected capacity change from 0 to 204800000 [ 740.424685] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 740.435181] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 740.891643] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 748.722827] loop0: detected capacity change from 0 to 204800000 [ 749.051968] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 749.092364] blk_update_request: operation not supported error, dev loop0, sector 8224 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 749.128748] blk_update_request: operation not supported error, dev loop0, sector 58264 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 754.971803] loop0: detected capacity change from 0 to 204800000 [ 761.647482] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 761.670429] LDISKFS-fs (dm-4): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 762.956971] Lustre: First target registration detected: opening a one-time 24-hour registration window. Use 'lctl set_param allow_register=0' to close it early, or 'lctl set_param -P allow_register=0' to lock down permanently. [ 762.979569] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 763.046758] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 763.191818] Lustre: lustre-MDT0000: new disk, initializing [ 763.345445] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 763.371283] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 769.146567] Lustre: DEBUG MARKER: oleg221-server.virtnet: executing set_default_debug -1 all [ 778.129111] loop1: detected capacity change from 0 to 204800000 [ 785.273253] LDISKFS-fs (dm-5): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 785.692977] Lustre: lustre-OST07c6: new disk, initializing [ 785.698497] Lustre: srv-lustre-OST07c6: No data found on store. Initialize space. [ 785.705214] Lustre: Skipped 1 previous similar message [ 785.709909] Lustre: 26197:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 785.814888] Lustre: lustre-OST07c6: Imperative Recovery not enabled, recovery window 60-180 [ 793.462549] Lustre: DEBUG MARKER: oleg221-server.virtnet: executing set_default_debug -1 all [ 795.804711] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:7c6:ost [ 795.818951] Lustre: cli-lustre-OST07c6-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:7c6:ost] [ 795.854837] Lustre: lustre-OST07c6-osc-MDT0000: update sequence from 0x107c60000 to 0x240000400 [ 799.650371] Lustre: 25182:0:(osd_internal.h:1460:osd_trans_exec_op()) lustre-MDT0000: opcode 2: before 254 < left 264, rollback = 2 [ 799.665097] Lustre: 25182:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 1/4/2, destroy: 0/0/0 [ 799.672690] Lustre: 25182:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 1/1/0, xattr_set: 3/264/0 [ 799.684976] Lustre: 25182:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 1/11/0, punch: 0/0/0, quota 1/3/0 [ 799.697574] Lustre: 25182:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 4/65/4, delete: 0/0/0 [ 799.705996] Lustre: 25182:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 2/2/0, ref_del: 0/0/0 [ 823.085193] Lustre: 25183:0:(osd_internal.h:1460:osd_trans_exec_op()) lustre-MDT0000: opcode 2: before 250 < left 278, rollback = 2 [ 823.094497] Lustre: 25183:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 1/4/4, destroy: 0/0/0 [ 823.099611] Lustre: 25183:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 1/1/0, xattr_set: 4/278/0 [ 823.105806] Lustre: 25183:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/20/1, punch: 0/0/0, quota 4/150/4 [ 823.110628] Lustre: 25183:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 2/33/1, delete: 0/0/0 [ 823.116494] Lustre: 25183:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 828.256790] Lustre: 25184:0:(osd_internal.h:1460:osd_trans_exec_op()) lustre-MDT0000: opcode 2: before 252 < left 278, rollback = 2 [ 828.262639] Lustre: 25184:0:(osd_internal.h:1460:osd_trans_exec_op()) Skipped 1 previous similar message [ 828.267310] Lustre: 25184:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 1/4/4, destroy: 0/0/0 [ 828.271790] Lustre: 25184:0:(osd_handler.c:2090:osd_trans_dump_creds()) Skipped 1 previous similar message [ 828.281400] Lustre: 25184:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 1/1/0, xattr_set: 4/278/0 [ 828.287283] Lustre: 25184:0:(osd_handler.c:2097:osd_trans_dump_creds()) Skipped 1 previous similar message [ 828.293103] Lustre: 25184:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/20/1, punch: 0/0/0, quota 4/150/2 [ 828.300336] Lustre: 25184:0:(osd_handler.c:2107:osd_trans_dump_creds()) Skipped 1 previous similar message [ 828.307642] Lustre: 25184:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 2/33/1, delete: 0/0/0 [ 828.317110] Lustre: 25184:0:(osd_handler.c:2114:osd_trans_dump_creds()) Skipped 1 previous similar message [ 828.324258] Lustre: 25184:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 828.331297] Lustre: 25184:0:(osd_handler.c:2121:osd_trans_dump_creds()) Skipped 1 previous similar message [ 831.529517] Lustre: 25183:0:(osd_internal.h:1460:osd_trans_exec_op()) lustre-MDT0000: opcode 2: before 260 < left 278, rollback = 2 [ 831.543204] Lustre: 25183:0:(osd_internal.h:1460:osd_trans_exec_op()) Skipped 1 previous similar message [ 831.549506] Lustre: 25183:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 1/4/0, destroy: 0/0/0 [ 831.557181] Lustre: 25183:0:(osd_handler.c:2090:osd_trans_dump_creds()) Skipped 1 previous similar message [ 831.562512] Lustre: 25183:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 1/1/0, xattr_set: 4/278/0 [ 831.569267] Lustre: 25183:0:(osd_handler.c:2097:osd_trans_dump_creds()) Skipped 1 previous similar message [ 831.579814] Lustre: 25183:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/20/0, punch: 0/0/0, quota 4/150/0 [ 831.590921] Lustre: 25183:0:(osd_handler.c:2107:osd_trans_dump_creds()) Skipped 1 previous similar message [ 831.601099] Lustre: 25183:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 2/33/0, delete: 0/0/0 [ 831.617117] Lustre: 25183:0:(osd_handler.c:2114:osd_trans_dump_creds()) Skipped 1 previous similar message [ 831.627595] Lustre: 25183:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 831.633204] Lustre: 25183:0:(osd_handler.c:2121:osd_trans_dump_creds()) Skipped 1 previous similar message [ 836.622817] Lustre: 25183:0:(osd_internal.h:1460:osd_trans_exec_op()) lustre-MDT0000: opcode 2: before 253 < left 278, rollback = 2 [ 836.636673] Lustre: 25183:0:(osd_internal.h:1460:osd_trans_exec_op()) Skipped 1 previous similar message [ 836.648282] Lustre: 25183:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 1/4/4, destroy: 0/0/0 [ 836.656649] Lustre: 25183:0:(osd_handler.c:2090:osd_trans_dump_creds()) Skipped 1 previous similar message [ 836.661591] Lustre: 25183:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 1/1/0, xattr_set: 4/278/0 [ 836.667770] Lustre: 25183:0:(osd_handler.c:2097:osd_trans_dump_creds()) Skipped 1 previous similar message [ 836.673163] Lustre: 25183:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/20/1, punch: 0/0/0, quota 4/150/2 [ 836.681322] Lustre: 25183:0:(osd_handler.c:2107:osd_trans_dump_creds()) Skipped 1 previous similar message [ 836.689876] Lustre: 25183:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 2/33/0, delete: 0/0/0 [ 836.694687] Lustre: 25183:0:(osd_handler.c:2114:osd_trans_dump_creds()) Skipped 1 previous similar message [ 836.699270] Lustre: 25183:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 836.704832] Lustre: 25183:0:(osd_handler.c:2121:osd_trans_dump_creds()) Skipped 1 previous similar message [ 845.042374] Lustre: 25184:0:(osd_internal.h:1460:osd_trans_exec_op()) lustre-MDT0000: opcode 2: before 253 < left 278, rollback = 2 [ 845.056484] Lustre: 25184:0:(osd_internal.h:1460:osd_trans_exec_op()) Skipped 3 previous similar messages [ 845.069354] Lustre: 25184:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 1/4/4, destroy: 0/0/0 [ 845.075828] Lustre: 25184:0:(osd_handler.c:2090:osd_trans_dump_creds()) Skipped 3 previous similar messages [ 845.081323] Lustre: 25184:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 1/1/0, xattr_set: 4/278/0 [ 845.089136] Lustre: 25184:0:(osd_handler.c:2097:osd_trans_dump_creds()) Skipped 3 previous similar messages [ 845.095388] Lustre: 25184:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/20/1, punch: 0/0/0, quota 4/150/2 [ 845.102251] Lustre: 25184:0:(osd_handler.c:2107:osd_trans_dump_creds()) Skipped 3 previous similar messages [ 845.117134] Lustre: 25184:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 2/33/0, delete: 0/0/0 [ 845.128818] Lustre: 25184:0:(osd_handler.c:2114:osd_trans_dump_creds()) Skipped 3 previous similar messages [ 845.141630] Lustre: 25184:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 845.151915] Lustre: 25184:0:(osd_handler.c:2121:osd_trans_dump_creds()) Skipped 3 previous similar messages [ 903.148597] Lustre: lustre-OST07c6-osc-MDT0000: Connection to lustre-OST07c6 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 903.156587] Lustre: lustre-OST07c6: Not available for connect from 0@lo (stopping) [ 905.627894] Lustre: server umount lustre-OST07c6 complete [ 916.352753] Lustre: server umount lustre-MDT0000 complete [ 933.840809] Lustre: DEBUG MARKER: oleg221-server.virtnet: executing unload_modules_local [ 937.155329] Key type lgssc unregistered [ 937.554812] LNet: 28759:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 937.558806] LNetError: 28759:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 937.585234] LNet: Removed LNI 192.168.202.121@tcp [ 938.669204] Key type .llcrypt unregistered [ 938.673343] Key type ._llcrypt unregistered [ 966.879244] Key type ._llcrypt registered [ 966.880978] Key type .llcrypt registered [ 967.083291] Lustre: DEBUG MARKER: oleg221-server.virtnet: executing load_modules_local [ 968.914186] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 969.339585] alg: No test for adler32 (adler32-zlib) [ 970.794731] Lustre: Lustre: Build Version: 2.17.58_2_gefa5002 [ 971.006498] LNet: Added LNI 192.168.202.121@tcp [8/256/0/180] [ 972.666044] Key type lgssc registered [ 974.246528] Lustre: Echo OBD driver; http://www.lustre.org/ [ 983.584758] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 983.616438] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 984.889809] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 989.381283] Lustre: DEBUG MARKER: oleg221-server.virtnet: executing set_default_debug -1 all [ 995.297867] LustreError: 30026:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 995.325610] LustreError: 30026:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 999.685866] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 999.903545] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 1005.593524] Lustre: DEBUG MARKER: oleg221-server.virtnet: executing set_default_debug -1 all [ 1014.744428] Lustre: DEBUG MARKER: oleg221-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1021.720331] Lustre: DEBUG MARKER: oleg221-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 1029.312707] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1029.605402] LustreError: 31850:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1029.716220] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1034.756896] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:35 to 0x280000401:65) [ 1037.400128] Lustre: DEBUG MARKER: oleg221-server.virtnet: executing set_default_debug -1 all [ 1046.736443] Lustre: DEBUG MARKER: oleg221-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1065.440267] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1065.449223] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1065.460624] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1065.487937] Lustre: Skipped 1 previous similar message [ 1070.575456] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1070.590441] Lustre: Skipped 1 previous similar message [ 1075.681345] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1075.690150] Lustre: Skipped 2 previous similar messages [ 1079.263149] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 1079.641459] Lustre: server umount lustre-OST0000 complete [ 1085.921743] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1085.949718] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1085.954586] Lustre: Skipped 1 previous similar message [ 1091.041159] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1091.049583] Lustre: Skipped 1 previous similar message [ 1098.207667] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 1098.582846] Lustre: server umount lustre-MDT0000 complete [ 1101.281301] LustreError: 30025:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1101.300592] LustreError: 30025:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 1103.415706] LustreError: 30006:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788800336 with bad export cookie 3971704860904142030 [ 1103.445549] LustreError: MGC192.168.202.121@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1104.203339] Lustre: server umount lustre-MDT0001 complete [ 1118.306933] Lustre: DEBUG MARKER: oleg221-server.virtnet: executing unload_modules_local [ 1122.285066] Key type lgssc unregistered [ 1122.840988] LNet: 33767:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1122.849811] LNetError: 33767:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1122.871688] LNet: Removed LNI 192.168.202.121@tcp [ 1123.872583] Key type .llcrypt unregistered [ 1123.876911] Key type ._llcrypt unregistered [ 1147.689711] Key type ._llcrypt registered [ 1147.694625] Key type .llcrypt registered [ 1147.871974] Lustre: DEBUG MARKER: oleg221-server.virtnet: executing load_modules_local [ 1149.156319] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1149.241062] alg: No test for adler32 (adler32-zlib) [ 1150.306304] Lustre: Lustre: Build Version: 2.17.58_2_gefa5002 [ 1150.606613] LNet: Added LNI 192.168.202.121@tcp [8/256/0/180] [ 1152.447144] Key type lgssc registered [ 1154.159603] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1163.584811] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1163.618649] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1165.298441] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1171.141749] Lustre: DEBUG MARKER: oleg221-server.virtnet: executing set_default_debug -1 all [ 1174.519923] LustreError: 35039:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1174.546399] LustreError: 35039:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 1179.639215] LustreError: 35038:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1181.933622] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1182.518828] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 1188.281826] Lustre: DEBUG MARKER: oleg221-server.virtnet: executing set_default_debug -1 all [ 1197.312338] Lustre: DEBUG MARKER: oleg221-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1203.757841] Lustre: DEBUG MARKER: oleg221-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 1211.464705] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1212.101394] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1220.882822] Lustre: DEBUG MARKER: oleg221-server.virtnet: executing set_default_debug -1 all [ 1233.164487] Lustre: DEBUG MARKER: oleg221-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1234.621323] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:67 to 0x280000401:97) [ 1245.162561] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1245.165652] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1245.187720] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1245.198727] Lustre: Skipped 1 previous similar message [ 1249.875938] Lustre: server umount lustre-OST0000 complete [ 1254.378512] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1254.392975] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1254.420062] Lustre: Skipped 1 previous similar message [ 1254.431689] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1255.394604] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1260.481600] Lustre: server umount lustre-MDT0000 complete [ 1260.527148] LustreError: 35038:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1260.543882] LustreError: 35038:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 1264.613402] LustreError: 37548:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1264.630420] LustreError: 37548:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 1265.285603] LustreError: 35017:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788800498 with bad export cookie 13067233542468632268 [ 1265.297285] LustreError: 35017:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 1265.304211] LustreError: MGC192.168.202.121@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1265.708418] Lustre: server umount lustre-MDT0001 complete [ 1279.078202] Lustre: DEBUG MARKER: oleg221-server.virtnet: executing unload_modules_local [ 1281.774967] Key type lgssc unregistered [ 1282.034990] LNet: 38589:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1282.050628] LNetError: 38589:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1282.074226] LNet: Removed LNI 192.168.202.121@tcp [ 1282.927306] Key type .llcrypt unregistered [ 1282.928992] Key type ._llcrypt unregistered [ 1309.718542] Key type ._llcrypt registered [ 1309.721464] Key type .llcrypt registered [ 1309.845742] Lustre: DEBUG MARKER: oleg221-server.virtnet: executing load_modules_local [ 1311.542234] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1311.572428] alg: No test for adler32 (adler32-zlib) [ 1312.702718] Lustre: Lustre: Build Version: 2.17.58_2_gefa5002 [ 1313.099460] LNet: Added LNI 192.168.202.121@tcp [8/256/0/180] [ 1314.786516] Key type lgssc registered [ 1316.404439] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1325.610519] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1325.621756] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1327.259479] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1332.211186] Lustre: DEBUG MARKER: oleg221-server.virtnet: executing set_default_debug -1 all [ 1336.811210] LustreError: 40001:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1336.834417] LustreError: 40001:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 1341.921507] LustreError: 40000:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1342.219579] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1342.585524] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 1347.497621] Lustre: DEBUG MARKER: oleg221-server.virtnet: executing set_default_debug -1 all [ 1355.293332] Lustre: DEBUG MARKER: oleg221-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1361.316312] Lustre: DEBUG MARKER: oleg221-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 1367.963490] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1368.329290] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1372.409398] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:99 to 0x280000401:129) [ 1374.712150] Lustre: DEBUG MARKER: oleg221-server.virtnet: executing set_default_debug -1 all [ 1383.979314] Lustre: DEBUG MARKER: oleg221-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1392.228542] Lustre: lustre-MDT0000: Not available for connect from 192.168.202.21@tcp (stopping) [ 1393.121034] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1393.140867] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1393.143594] Lustre: Skipped 2 previous similar messages [ 1393.159074] Lustre: Skipped 2 previous similar messages [ 1397.314650] Lustre: lustre-MDT0000: Not available for connect from 192.168.202.21@tcp (stopping) [ 1397.766691] Lustre: server umount lustre-MDT0000 complete [ 1398.246614] LustreError: 42581:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1398.287774] LustreError: 42581:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 1402.264685] LustreError: 39981:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788800635 with bad export cookie 8948040203495266502 [ 1402.266382] LustreError: MGC192.168.202.121@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1402.275257] LustreError: 39981:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 1402.443141] LustreError: 41184:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 192.168.202.21@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1402.486633] Lustre: lustre-MDT0001: Not available for connect from 192.168.202.21@tcp (stopping) [ 1403.362231] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1407.394530] LustreError: 39995:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1407.395025] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 1407.412866] LustreError: 39995:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 1407.422892] Lustre: Skipped 1 previous similar message [ 1408.743921] Lustre: server umount lustre-MDT0001 complete [ 1429.686794] Lustre: server umount lustre-OST0000 complete [ 1446.160600] Lustre: DEBUG MARKER: oleg221-server.virtnet: executing unload_modules_local [ 1449.001393] Key type lgssc unregistered [ 1449.436224] LNet: 43859:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1449.445816] LNetError: 43859:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1449.485695] LNet: Removed LNI 192.168.202.121@tcp [ 1450.779627] Key type .llcrypt unregistered [ 1450.782373] Key type ._llcrypt unregistered [ 1482.685813] Key type ._llcrypt registered [ 1482.692299] Key type .llcrypt registered [ 1482.849994] Lustre: DEBUG MARKER: oleg221-server.virtnet: executing load_modules_local [ 1484.207091] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1484.235094] alg: No test for adler32 (adler32-zlib) [ 1485.426301] Lustre: Lustre: Build Version: 2.17.58_2_gefa5002 [ 1485.833493] LNet: Added LNI 192.168.202.121@tcp [8/256/0/180] [ 1487.647185] Key type lgssc registered [ 1488.705984] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1496.920309] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1496.935825] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1498.406196] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1503.577948] Lustre: DEBUG MARKER: oleg221-server.virtnet: executing set_default_debug -1 all [ 1507.813955] LustreError: 45287:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1507.832353] LustreError: 45287:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 1512.155075] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1512.606293] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 1517.325792] Lustre: DEBUG MARKER: oleg221-server.virtnet: executing set_default_debug -1 all [ 1525.322208] Lustre: DEBUG MARKER: oleg221-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1531.489981] Lustre: DEBUG MARKER: oleg221-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 1538.895741] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1539.353048] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1543.468718] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:131 to 0x280000401:161) [ 1546.680607] Lustre: DEBUG MARKER: oleg221-server.virtnet: executing set_default_debug -1 all [ 1554.693545] Lustre: DEBUG MARKER: oleg221-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1567.869285] Lustre: lustre-OST0000: Not available for connect from 192.168.202.21@tcp (stopping) [ 1568.229029] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 1568.231373] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1571.220330] Lustre: server umount lustre-OST0000 complete [ 1589.216409] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1589.227598] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1589.246732] Lustre: Skipped 1 previous similar message [ 1589.256687] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1589.265711] Lustre: Skipped 2 previous similar messages [ 1592.802212] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1592.815237] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1595.352049] Lustre: server umount lustre-MDT0000 complete [ 1597.920280] LustreError: 45288:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1597.940028] LustreError: 45288:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 1599.247756] LustreError: 45269:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788800832 with bad export cookie 18300773099749288198 [ 1599.257150] LustreError: MGC192.168.202.121@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1599.261417] LustreError: 45269:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 1605.710639] Lustre: server umount lustre-MDT0001 complete [ 1616.508225] Lustre: DEBUG MARKER: oleg221-server.virtnet: executing unload_modules_local [ 1619.192874] Key type lgssc unregistered [ 1619.545566] LNet: 48995:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1619.556395] LNetError: 48995:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1619.581253] LNet: Removed LNI 192.168.202.121@tcp [ 1620.582897] Key type .llcrypt unregistered [ 1620.584710] Key type ._llcrypt unregistered [ 1646.177638] Key type ._llcrypt registered [ 1646.180644] Key type .llcrypt registered [ 1646.327286] Lustre: DEBUG MARKER: oleg221-server.virtnet: executing load_modules_local [ 1647.463200] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1647.517650] alg: No test for adler32 (adler32-zlib) [ 1648.679356] Lustre: Lustre: Build Version: 2.17.58_2_gefa5002 [ 1648.972823] LNet: Added LNI 192.168.202.121@tcp [8/256/0/180] [ 1650.697353] Key type lgssc registered [ 1651.953745] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1660.703886] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1660.732043] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1662.270350] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1667.013344] Lustre: DEBUG MARKER: oleg221-server.virtnet: executing set_default_debug -1 all [ 1671.676670] LustreError: 50410:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1671.710264] LustreError: 50410:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 1676.775640] LustreError: 50411:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1677.853912] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1678.389303] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 1684.784122] Lustre: DEBUG MARKER: oleg221-server.virtnet: executing set_default_debug -1 all [ 1696.264550] Lustre: DEBUG MARKER: oleg221-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1703.445751] Lustre: DEBUG MARKER: oleg221-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 1712.454233] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1712.837640] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1719.912337] Lustre: DEBUG MARKER: oleg221-server.virtnet: executing set_default_debug -1 all [ 1729.117687] Lustre: DEBUG MARKER: oleg221-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1732.349369] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:163 to 0x280000401:193) [ 1739.125900] Lustre: DEBUG MARKER: Set up a fake failnode for the MDS [ 1742.257226] Lustre: DEBUG MARKER: Wait for RECONNECT_INTERVAL seconds (10s) [ 1754.627269] Lustre: DEBUG MARKER: conf-sanity.sh test_35a 2026-09-0713h09m46s [ 1756.464033] Lustre: DEBUG MARKER: Stopping the MDT: lustre-MDT0000 [ 1760.226663] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1760.247937] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1760.260908] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1761.364673] Lustre: lustre-MDT0000: Not available for connect from 192.168.202.21@tcp (stopping) [ 1761.367397] Lustre: Skipped 2 previous similar messages [ 1762.448906] Lustre: lustre-MDT0000: Not available for connect from 192.168.202.21@tcp (stopping) [ 1765.020986] Lustre: server umount lustre-MDT0000 complete [ 1767.511826] LustreError: 50407:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 192.168.202.21@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1769.209504] Lustre: DEBUG MARKER: Restarting the MDT: lustre-MDT0000 [ 1773.636516] LustreError: 50407:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 192.168.202.21@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1773.694371] LustreError: 50407:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [ 1777.639648] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1777.798654] LustreError: MGC192.168.202.121@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1778.228769] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1778.332670] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:163 to 0x280000401:225) [ 1779.262187] Lustre: lustre-MDT0000-lwp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 1784.352096] Lustre: DEBUG MARKER: oleg221-server.virtnet: executing set_default_debug -1 all [ 1788.628540] Lustre: DEBUG MARKER: Wait for df (36716) ... [ 1790.353527] Lustre: DEBUG MARKER: done [ 1798.635299] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1798.648455] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1798.668240] Lustre: Skipped 2 previous similar messages [ 1798.678942] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1799.658713] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1804.774744] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1804.788948] Lustre: Skipped 2 previous similar messages [ 1809.375141] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 1809.589602] Lustre: server umount lustre-OST0000 complete [ 1815.017449] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1815.037107] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1815.049291] Lustre: Skipped 1 previous similar message [ 1815.073231] Lustre: Skipped 2 previous similar messages [ 1820.417913] Lustre: server umount lustre-MDT0000 complete [ 1822.202361] LustreError: 50407:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1822.220708] LustreError: 50407:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 4 previous similar messages [ 1826.093364] LustreError: 51150:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788801059 with bad export cookie 16271898625348854026 [ 1826.108366] LustreError: MGC192.168.202.121@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1826.113613] LustreError: 51150:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 1826.627360] Lustre: server umount lustre-MDT0001 complete [ 1843.070856] Lustre: DEBUG MARKER: oleg221-server.virtnet: executing unload_modules_local [ 1846.429165] Key type lgssc unregistered [ 1846.763671] LNet: 55336:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1846.771410] LNetError: 55336:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1846.784289] LNet: Removed LNI 192.168.202.121@tcp [ 1847.839520] Key type .llcrypt unregistered [ 1847.846423] Key type ._llcrypt unregistered [ 1893.526698] Key type ._llcrypt registered [ 1893.528800] Key type .llcrypt registered [ 1893.681654] Lustre: DEBUG MARKER: oleg221-server.virtnet: executing load_modules_local [ 1894.973209] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1895.190459] alg: No test for adler32 (adler32-zlib) [ 1896.388654] Lustre: Lustre: Build Version: 2.17.58_2_gefa5002 [ 1896.666573] LNet: Added LNI 192.168.202.121@tcp [8/256/0/180] [ 1898.320508] Key type lgssc registered [ 1899.487915] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1910.968958] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1910.993158] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1912.337190] Lustre: MGS: Logs for fs lustre were removed by user request. All servers must be restarted in order to regenerate the logs: rc = 0 [ 1912.348858] Lustre: First target registration detected: opening a one-time 24-hour registration window. Use 'lctl set_param allow_register=0' to close it early, or 'lctl set_param -P allow_register=0' to lock down permanently. [ 1912.417436] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 1912.693896] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1918.242559] Lustre: DEBUG MARKER: oleg221-server.virtnet: executing set_default_debug -1 all [ 1929.100925] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1929.331856] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [ 1929.365690] Lustre: 57544:0:(mgs_llog.c:1462:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 1930.030481] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 1936.516581] Lustre: DEBUG MARKER: oleg221-server.virtnet: executing set_default_debug -1 all [ 1948.417207] Lustre: DEBUG MARKER: oleg221-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1955.305867] Lustre: DEBUG MARKER: oleg221-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 1964.689334] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1965.012759] Lustre: MGS: Regenerating lustre-OST0000 log by user request: rc = 0 [ 1965.543190] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1974.008171] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:163 to 0x280000401:257) [ 1974.074948] Lustre: DEBUG MARKER: oleg221-server.virtnet: executing set_default_debug -1 all [ 1983.835669] Lustre: DEBUG MARKER: oleg221-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1991.462523] Lustre: DEBUG MARKER: conf-sanity.sh test_35b 2026-09-0713h13m43s [ 1993.735232] Lustre: DEBUG MARKER: Set up a fake failnode for the MDS [ 2002.586890] Lustre: DEBUG MARKER: Injecting EBUSY on MDS [ 2005.462950] Lustre: DEBUG MARKER: Creating a test file and stat it [ 2005.510250] LustreError: 57552:0:(mdt_handler.c:2344:mdt_getattr_name_lock()) cfs_fail_timeout id 136 sleeping for 40000ms [ 2025.582493] Lustre: lustre-MDT0000: Client 370cda2b-d6f7-49d5-ad21-ea41aaf7846f (at 192.168.202.21@tcp) reconnecting [ 2045.575131] LustreError: 57552:0:(mdt_handler.c:2344:mdt_getattr_name_lock()) cfs_fail_timeout id 136 awake [ 2045.685905] Lustre: lustre-MDT0000: Client 370cda2b-d6f7-49d5-ad21-ea41aaf7846f (at 192.168.202.21@tcp) reconnecting [ 2048.277983] Lustre: DEBUG MARKER: Stop injecting EBUSY on MDS [ 2051.032932] Lustre: DEBUG MARKER: done [ 2061.285094] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2061.290881] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2061.292100] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2061.313718] Lustre: Skipped 1 previous similar message [ 2061.351452] LustreError: Skipped 1 previous similar message [ 2064.761651] Lustre: server umount lustre-OST0000 complete [ 2071.520791] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2071.525439] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2071.566663] Lustre: Skipped 1 previous similar message [ 2071.596707] Lustre: Skipped 2 previous similar messages [ 2072.612826] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2072.620679] Lustre: Skipped 1 previous similar message [ 2075.657531] Lustre: server umount lustre-MDT0000 complete [ 2080.233774] LustreError: 57535:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788801313 with bad export cookie 3506960627542532132 [ 2080.239449] LustreError: MGC192.168.202.121@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2080.257144] LustreError: 57535:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2080.691495] Lustre: server umount lustre-MDT0001 complete [ 2095.932790] Lustre: DEBUG MARKER: oleg221-server.virtnet: executing unload_modules_local [ 2099.315128] Key type lgssc unregistered [ 2099.623803] LNet: 61880:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2099.640699] LNetError: 61880:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2099.667068] LNet: Removed LNI 192.168.202.121@tcp [ 2100.558990] Key type .llcrypt unregistered [ 2100.562099] Key type ._llcrypt unregistered [ 2157.360460] Key type ._llcrypt registered [ 2157.363544] Key type .llcrypt registered [ 2157.543618] Lustre: DEBUG MARKER: oleg221-server.virtnet: executing load_modules_local [ 2158.945645] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2158.968551] alg: No test for adler32 (adler32-zlib) [ 2160.186523] Lustre: Lustre: Build Version: 2.17.58_2_gefa5002 [ 2160.616538] LNet: Added LNI 192.168.202.121@tcp [8/256/0/180] [ 2162.296139] Key type lgssc registered [ 2163.899073] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2168.132629] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 2168.144750] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2169.439760] Lustre: MGS: Logs for fs lustre were removed by user request. All servers must be restarted in order to regenerate the logs: rc = 0 [ 2169.456712] Lustre: First target registration detected: opening a one-time 24-hour registration window. Use 'lctl set_param allow_register=0' to close it early, or 'lctl set_param -P allow_register=0' to lock down permanently. [ 2169.495984] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2169.671643] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2171.271203] Lustre: Failing over lustre-MDT0000 [ 2171.795576] Lustre: server umount lustre-MDT0000 complete [ 2180.527640] Lustre: DEBUG MARKER: == conf-sanity test 38: MDS recreates missing lov_objid file from OST data ========================================================== 13:16:52 (1788801412) [ 2189.559798] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2190.064402] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2195.618943] Lustre: DEBUG MARKER: oleg221-server.virtnet: executing set_default_debug -1 all [ 2205.707408] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2205.903194] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [ 2205.971174] Lustre: 64689:0:(mgs_llog.c:1462:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2206.404807] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2211.726610] Lustre: DEBUG MARKER: oleg221-server.virtnet: executing set_default_debug -1 all [ 2221.891360] Lustre: DEBUG MARKER: oleg221-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2228.911248] Lustre: DEBUG MARKER: oleg221-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2237.364320] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2237.684841] Lustre: MGS: Regenerating lustre-OST0000 log by user request: rc = 0 [ 2237.978308] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2245.799335] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:259 to 0x280000401:289) [ 2245.869875] Lustre: DEBUG MARKER: oleg221-server.virtnet: executing set_default_debug -1 all [ 2255.644976] Lustre: DEBUG MARKER: oleg221-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2262.295953] Lustre: DEBUG MARKER: copying 10 files to /mnt/lustre/d38.conf-sanity [ 2271.713606] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2271.716099] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2271.728973] Lustre: Skipped 1 previous similar message [ 2276.070083] Lustre: server umount lustre-MDT0000 complete [ 2280.696657] LustreError: 65435:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788801514 with bad export cookie 8784188383280774332 [ 2280.700777] LustreError: MGC192.168.202.121@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2280.710223] LustreError: 65435:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 2281.194981] Lustre: server umount lustre-MDT0001 complete [ 2285.746738] Lustre: DEBUG MARKER: delete lov_objid file on MDS [ 2291.801547] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: (null) [ 2298.335248] Lustre: 63597:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1788801515/real 1788801515] req@00000000aaacb1f7 x1875694210558336/t0(0) o400->lustre-MDT0001-lwp-OST0000@0@lo:12/10 lens 224/224 e 0 to 1 dl 1788801531 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 2298.368337] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2298.383324] Lustre: Skipped 1 previous similar message [ 2302.211488] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2302.433621] Lustre: 63598:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1788801519/real 1788801519] req@000000009efeb144 x1875694210558464/t0(0) o400->lustre-MDT0001-lwp-OST0000@0@lo:12/10 lens 224/224 e 0 to 1 dl 1788801535 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 2303.455544] Lustre: 63600:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1788801520/real 1788801520] req@000000000c1a1520 x1875694210558592/t0(0) o400->lustre-MDT0001-lwp-OST0000@0@lo:12/10 lens 224/224 e 0 to 1 dl 1788801536 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 2306.019482] LustreError: 66538:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2306.046758] LustreError: 66538:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 4 previous similar messages [ 2306.157740] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2306.289936] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000402 [ 2307.167086] Lustre: 63600:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1788801524/real 1788801524] req@00000000d996659a x1875694210558848/t0(0) o400->lustre-MDT0001-lwp-OST0000@0@lo:12/10 lens 224/224 e 0 to 1 dl 1788801540 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 2308.201519] LustreError: 66539:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2308.225690] LustreError: 66539:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 2311.363757] Lustre: DEBUG MARKER: oleg221-server.virtnet: executing set_default_debug -1 all [ 2312.161285] Lustre: 63598:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1788801529/real 1788801529] req@00000000fd65c25e x1875694210558976/t0(0) o400->lustre-MDT0001-lwp-OST0000@0@lo:12/10 lens 224/224 e 0 to 1 dl 1788801545 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 2313.703310] LustreError: 68460:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2313.742521] LustreError: 68460:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 2318.816477] LustreError: 68459:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2318.854458] LustreError: 68459:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 2321.416179] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2321.886458] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2326.867512] Lustre: DEBUG MARKER: oleg221-server.virtnet: executing set_default_debug -1 all [ 2327.039898] Lustre: lustre-MDT0000-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 2335.842870] Lustre: DEBUG MARKER: oleg221-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2342.186767] Lustre: DEBUG MARKER: oleg221-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2352.612651] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2352.618978] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2352.619873] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2352.619878] Lustre: Skipped 2 previous similar messages [ 2352.631282] Lustre: Skipped 1 previous similar message [ 2356.912590] Lustre: server umount lustre-MDT0000 complete [ 2357.733276] LustreError: 68460:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2357.770177] LustreError: 68460:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 2361.725931] LustreError: 64679:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788801595 with bad export cookie 8784188383280776607 [ 2361.739265] LustreError: MGC192.168.202.121@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2361.741457] LustreError: 64679:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 2362.166496] Lustre: server umount lustre-MDT0001 complete [ 2370.581560] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: (null) [ 2379.233059] Lustre: 63598:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1788801596/real 1788801596] req@000000007cd471b8 x1875694210585984/t0(0) o400->lustre-MDT0001-lwp-OST0000@0@lo:12/10 lens 224/224 e 0 to 1 dl 1788801612 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 2379.288437] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2379.311622] Lustre: Skipped 1 previous similar message [ 2382.375724] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2387.003932] LustreError: 66538:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2387.113574] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2387.203775] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000402:3 to 0x280000402:33) [ 2393.756644] Lustre: DEBUG MARKER: oleg221-server.virtnet: executing set_default_debug -1 all [ 2404.847062] LustreError: 71144:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2404.863573] LustreError: 71144:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 7 previous similar messages [ 2405.282379] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2407.715212] Lustre: lustre-MDT0000-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 2407.729580] Lustre: Skipped 1 previous similar message [ 2411.054219] Lustre: DEBUG MARKER: oleg221-server.virtnet: executing set_default_debug -1 all [ 2420.480902] Lustre: DEBUG MARKER: oleg221-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2427.200285] Lustre: DEBUG MARKER: oleg221-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2436.577944] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2436.589819] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2436.607059] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2436.614043] Lustre: Skipped 2 previous similar messages [ 2438.627780] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2443.747746] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2443.757454] Lustre: Skipped 4 previous similar messages [ 2450.401038] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2450.563347] Lustre: server umount lustre-MDT0000 complete [ 2453.985555] LustreError: 72498:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2454.007962] LustreError: 72498:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 2454.815199] LustreError: 64680:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788801688 with bad export cookie 8784188383280777986 [ 2454.826138] LustreError: MGC192.168.202.121@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2454.833851] LustreError: 64680:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2455.255377] Lustre: server umount lustre-MDT0001 complete [ 2462.216940] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: (null) [ 2466.436335] Lustre: DEBUG MARKER: files compared the same [ 2479.240502] Lustre: server umount lustre-OST0000 complete [ 2497.831571] Lustre: DEBUG MARKER: oleg221-server.virtnet: executing unload_modules_local [ 2500.579864] Key type lgssc unregistered [ 2500.855929] LNet: 74422:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2500.861786] LNetError: 74422:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2500.877464] LNet: Removed LNI 192.168.202.121@tcp [ 2501.875950] Key type .llcrypt unregistered [ 2501.881591] Key type ._llcrypt unregistered [ 2531.020484] Key type ._llcrypt registered [ 2531.022945] Key type .llcrypt registered [ 2531.128995] Lustre: DEBUG MARKER: oleg221-server.virtnet: executing load_modules_local [ 2532.541038] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2532.558687] alg: No test for adler32 (adler32-zlib) [ 2533.576677] Lustre: Lustre: Build Version: 2.17.58_2_gefa5002 [ 2533.977419] LNet: Added LNI 192.168.202.121@tcp [8/256/0/180] [ 2535.664848] Key type lgssc registered [ 2537.422864] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2547.259193] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 2547.277164] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2548.925560] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2556.765578] Lustre: DEBUG MARKER: oleg221-server.virtnet: executing set_default_debug +malloc all [ 2559.461398] LustreError: 75837:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2559.478696] LustreError: 75837:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 2564.578972] LustreError: 75838:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2567.305532] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2567.506000] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2573.121752] Lustre: DEBUG MARKER: oleg221-server.virtnet: executing set_default_debug +malloc all [ 2581.935957] Lustre: DEBUG MARKER: oleg221-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2588.788960] Lustre: DEBUG MARKER: oleg221-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2596.084432] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2596.301960] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2603.056342] Lustre: DEBUG MARKER: oleg221-server.virtnet: executing set_default_debug +malloc all [ 2611.637749] Lustre: DEBUG MARKER: oleg221-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2619.853808] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000402:35 to 0x280000402:65) [ 2633.186428] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2633.202455] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2633.216416] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2635.232185] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2635.237949] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2635.243652] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2635.293558] Lustre: Skipped 1 previous similar message [ 2637.442333] Lustre: server umount lustre-OST0000 complete [ 2644.448047] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2644.452900] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2644.486160] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2648.052871] Lustre: server umount lustre-MDT0000 complete [ 2650.593486] LustreError: 76795:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2650.616278] LustreError: 76795:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 2652.006247] LustreError: 75819:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788801885 with bad export cookie 17103239976133144502 [ 2652.019388] LustreError: MGC192.168.202.121@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2652.407304] Lustre: server umount lustre-MDT0001 complete [ 2666.407120] Lustre: DEBUG MARKER: oleg221-server.virtnet: executing unload_modules_local [ 2669.722537] Key type lgssc unregistered [ 2670.070723] LNet: 79394:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2670.078521] LNetError: 79394:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2670.105185] LNet: Removed LNI 192.168.202.121@tcp [ 2671.003133] Key type .llcrypt unregistered [ 2671.006092] Key type ._llcrypt unregistered [ 2701.355342] Key type ._llcrypt registered [ 2701.357972] Key type .llcrypt registered [ 2701.502310] Lustre: DEBUG MARKER: oleg221-server.virtnet: executing load_modules_local [ 2702.694711] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2702.719696] alg: No test for adler32 (adler32-zlib) [ 2704.016348] Lustre: Lustre: Build Version: 2.17.58_2_gefa5002 [ 2704.348673] LNet: Added LNI 192.168.202.121@tcp [8/256/0/180] [ 2706.167193] Key type lgssc registered [ 2707.881868] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2718.127035] Lustre: lustre-OST0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 2718.142173] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2733.663873] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 2738.795127] LustreError: 80784:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.202.121@tcp: failed processing log, type 4: rc = -110 [ 2765.535218] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2773.243068] Lustre: DEBUG MARKER: oleg221-server.virtnet: executing set_default_debug -1 all [ 2782.140584] Lustre: DEBUG MARKER: oleg221-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2789.397606] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2789.547703] Lustre: *** cfs_fail_loc=706, val=0*** [ 2790.065614] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2790.159942] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000402:35 to 0x280000402:97) [ 2794.685401] Lustre: DEBUG MARKER: oleg221-server.virtnet: executing set_default_debug -1 all [ 2795.493926] LustreError: 80809:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2795.520802] LustreError: 80809:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [ 2800.614124] LustreError: 81869:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2800.639181] LustreError: 81869:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 2804.548664] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2805.088870] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2809.991889] Lustre: DEBUG MARKER: oleg221-server.virtnet: executing set_default_debug -1 all [ 2820.225179] Lustre: DEBUG MARKER: oleg221-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2827.729950] Lustre: DEBUG MARKER: oleg221-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2830.817436] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2830.825266] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2830.829379] Lustre: Skipped 1 previous similar message [ 2830.842198] Lustre: Skipped 1 previous similar message [ 2835.942582] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2835.956718] Lustre: Skipped 1 previous similar message [ 2836.613313] Lustre: server umount lustre-OST0000 complete [ 2841.058338] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2841.059670] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2841.103896] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2841.111214] Lustre: Skipped 1 previous similar message [ 2846.184139] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2846.187086] Lustre: Skipped 1 previous similar message [ 2846.757795] Lustre: server umount lustre-MDT0000 complete [ 2850.722886] LustreError: 83649:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788802084 with bad export cookie 7706928833385935857 [ 2850.728590] LustreError: MGC192.168.202.121@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2850.748896] LustreError: 83649:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2851.278484] Lustre: server umount lustre-MDT0001 complete [ 2864.270349] Lustre: DEBUG MARKER: oleg221-server.virtnet: executing unload_modules_local [ 2868.095411] Key type lgssc unregistered [ 2868.351256] LNet: 84400:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2868.359964] LNetError: 84400:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2868.375367] LNet: Removed LNI 192.168.202.121@tcp [ 2869.514908] Key type .llcrypt unregistered [ 2869.517224] Key type ._llcrypt unregistered [ 2902.383462] Key type ._llcrypt registered [ 2902.385771] Key type .llcrypt registered [ 2902.519848] Lustre: DEBUG MARKER: oleg221-server.virtnet: executing load_modules_local [ 2903.855778] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2904.123454] alg: No test for adler32 (adler32-zlib) [ 2905.213646] Lustre: Lustre: Build Version: 2.17.58_2_gefa5002 [ 2905.515506] LNet: Added LNI 192.168.202.121@tcp [8/256/0/180] [ 2907.391289] Key type lgssc registered [ 2909.048835] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2918.907711] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 2918.946848] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2925.499415] Lustre: DEBUG MARKER: oleg221-server.virtnet: executing set_default_debug -1 all [ 2934.518437] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2935.079355] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2940.406452] LustreError: 86503:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2940.423781] LustreError: 86503:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [ 2940.522996] Lustre: DEBUG MARKER: oleg221-server.virtnet: executing set_default_debug -1 all [ 2944.488515] LustreError: 86506:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2944.500398] LustreError: 86506:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 2949.203965] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2949.597417] LustreError: 86503:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2949.625478] LustreError: 86503:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 2949.641017] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2954.728750] LustreError: 86501:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2954.759221] LustreError: 86501:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 2956.665358] Lustre: DEBUG MARKER: oleg221-server.virtnet: executing set_default_debug -1 all [ 2959.842328] LustreError: 86506:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2959.863571] LustreError: 86506:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 2964.942456] LustreError: MGC192.168.202.121@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2964.951742] Lustre: MGS: Client 910814a3-8694-470c-91f0-28d262040e6f (at 0@lo) reconnecting [ 2964.968558] LustreError: 86517:0:(mgs_handler.c:244:mgs_revoke_lock()) MGS: can't take cfg lock for 0x65727473756c/0x2 : rc = -11 [ 2965.191513] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2965.259221] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000402:35 to 0x280000402:129) [ 2970.451867] Lustre: DEBUG MARKER: oleg221-server.virtnet: executing set_default_debug -1 all [ 2985.953033] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2985.955729] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2985.980059] Lustre: Skipped 1 previous similar message [ 2985.993889] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2990.991148] Lustre: server umount lustre-OST0000 complete [ 2996.192487] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2996.200942] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2996.225415] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2996.243737] Lustre: Skipped 1 previous similar message [ 2996.254366] Lustre: Skipped 2 previous similar messages [ 3001.312921] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3001.329799] Lustre: server umount lustre-MDT0000 complete [ 3006.209497] Lustre: server umount lustre-MDT0001 complete [ 3010.790438] Lustre: server umount MGS complete [ 3025.976239] Lustre: DEBUG MARKER: == conf-sanity test 41b: mount mds with --nosvc and --nomgs on first mount ========================================================== 13:30:57 (1788802257) [ 3061.756696] Lustre: DEBUG MARKER: oleg221-server.virtnet: executing set_hostid [ 3073.457385] Lustre: DEBUG MARKER: oleg221-server.virtnet: executing load_modules_local [ 3114.478839] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3119.562501] Lustre: DEBUG MARKER: oleg221-server.virtnet: executing set_default_debug -1 all [ 3130.048285] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3130.361178] Lustre: First target registration detected: opening a one-time 24-hour registration window. Use 'lctl set_param allow_register=0' to close it early, or 'lctl set_param -P allow_register=0' to lock down permanently. [ 3130.417510] Lustre: Setting parameter lustre-MDT0001.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0001 [ 3130.468522] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 3130.552675] Lustre: lustre-MDT0001: new disk, initializing [ 3130.661744] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3130.706893] Lustre: srv-lustre-MDT0001: Waiting to contact MDT0000 to allocate super-sequence: rc = -115 [ 3130.725968] Lustre: 93696:0:(lod_sub_object.c:951:lod_sub_prep_llog()) lustre-MDT0001-mdtlov: can't create new llog: rc = -115 [ 3130.742920] LustreError: 93696:0:(lod_dev.c:511:lod_sub_recovery_thread()) lustre-MDT0001-osd: get update log duration 0, retries 0, failed: rc = -115 [ 3135.473569] Lustre: DEBUG MARKER: oleg221-server.virtnet: executing set_default_debug -1 all [ 3146.271627] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3146.575709] Lustre: lustre-OST0000: new disk, initializing [ 3146.580968] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3146.591427] Lustre: 94462:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3146.687139] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3154.345630] Lustre: DEBUG MARKER: oleg221-server.virtnet: executing set_default_debug -1 all [ 3157.088138] Lustre: 94884:0:(client.c:1652:after_reply()) @@@ resending request on EINPROGRESS req@00000000c5684957 x1875694991745536/t0(0) o700->lustre-OST0000-osc-MDT0001@0@lo:31/4 lens 264/248 e 0 to 0 dl 1788802406 ref 2 fl Rpc:RQU/202/0 rc 0/-115 job:'' uid:0 gid:0 projid:4294967295 [ 3167.111282] Lustre: DEBUG MARKER: oleg221-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3177.439984] Lustre: srv-lustre-OST0000: Waiting to contact MDT0000 to allocate super-sequence: rc = -115 [ 3177.894608] LustreError: MGC192.168.202.121@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3177.904723] Lustre: MGS: Client 80f90659-6994-4a39-8a95-1462fec17d5b (at 0@lo) reconnecting [ 3177.949777] Lustre: 92964:0:(mgs_llog.c:1462:mgs_modify_param()) MGS: modify lustre-MDT0000/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 3177.985921] LustreError: 93682:0:(mgs_handler.c:244:mgs_revoke_lock()) MGS: can't take cfg lock for 0x65727473756c/0x2 : rc = -11 [ 3178.000283] LustreError: 93682:0:(mgs_handler.c:244:mgs_revoke_lock()) Skipped 1 previous similar message [ 3178.015383] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3178.083275] Lustre: lustre-MDT0000: new disk, initializing [ 3178.235887] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3178.262492] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3179.296670] Lustre: 95531:0:(client.c:1652:after_reply()) @@@ resending request on EINPROGRESS req@000000002792ffe6 x1875694991752960/t0(0) o700->lustre-MDT0001-osp-MDT0000@0@lo:30/10 lens 264/248 e 0 to 0 dl 1788802428 ref 2 fl Rpc:RQU/202/0 rc 0/-115 job:'' uid:0 gid:0 projid:4294967295 [ 3184.486369] Lustre: srv-lustre-OST0000: Waiting to contact MDT0000 to allocate super-sequence: rc = -115 [ 3184.838895] Lustre: DEBUG MARKER: oleg221-server.virtnet: executing set_default_debug -1 all [ 3188.398570] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 3188.414598] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3188.569098] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000400 [ 3213.796276] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3213.811922] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3213.832090] Lustre: Skipped 1 previous similar message [ 3218.400632] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3218.408588] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3218.421757] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3228.128166] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 3228.601585] Lustre: server umount lustre-OST0000 complete [ 3234.272171] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3234.276983] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3234.285978] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3234.292471] Lustre: Skipped 1 previous similar message [ 3234.316639] Lustre: Skipped 4 previous similar messages [ 3239.145921] Lustre: server umount lustre-MDT0000 complete [ 3239.405570] LustreError: 93666:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3239.417356] LustreError: 93666:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [ 3258.337640] Lustre: lustre-MDT0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 3258.815498] Lustre: server umount lustre-MDT0001 complete [ 3264.389269] Lustre: server umount MGS complete [ 3278.072507] Lustre: DEBUG MARKER: == conf-sanity test 41c: concurrent mounts of MDT/OST should all fail but one ========================================================== 13:35:10 (1788802510) [ 3301.641083] Lustre: DEBUG MARKER: oleg221-server.virtnet: executing unload_modules_local [ 3305.089057] Key type lgssc unregistered [ 3305.523462] LNet: 98032:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3305.534365] LNetError: 98032:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3305.563564] LNet: Removed LNI 192.168.202.121@tcp [ 3306.680675] Key type .llcrypt unregistered [ 3306.684272] Key type ._llcrypt unregistered [ 3331.781981] Key type ._llcrypt registered [ 3331.783321] Key type .llcrypt registered [ 3331.921560] Lustre: DEBUG MARKER: oleg221-server.virtnet: executing load_modules_local [ 3333.837375] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3333.876456] alg: No test for adler32 (adler32-zlib) [ 3334.980579] Lustre: Lustre: Build Version: 2.17.58_2_gefa5002 [ 3335.231965] LNet: Added LNI 192.168.202.121@tcp [8/256/0/180] [ 3336.911188] Key type lgssc registered [ 3338.354788] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3347.985313] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3348.005549] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3349.674898] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3355.085937] Lustre: DEBUG MARKER: oleg221-server.virtnet: executing set_default_debug -1 all [ 3360.234648] LustreError: 99252:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3360.253121] LustreError: 99252:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 3365.347941] LustreError: 99253:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3365.909551] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3366.412166] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3371.542157] Lustre: DEBUG MARKER: oleg221-server.virtnet: executing set_default_debug -1 all [ 3383.459362] Lustre: DEBUG MARKER: oleg221-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3392.277936] Lustre: DEBUG MARKER: oleg221-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3401.539952] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3401.844030] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3410.425652] Lustre: DEBUG MARKER: oleg221-server.virtnet: executing set_default_debug -1 all [ 3420.293109] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000400:3 to 0x280000400:33) [ 3420.941437] Lustre: DEBUG MARKER: oleg221-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3430.881620] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3430.886219] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3430.896884] Lustre: Skipped 1 previous similar message [ 3430.907719] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3435.889762] Lustre: server umount lustre-OST0000 complete [ 3441.122695] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3441.131958] Lustre: Skipped 1 previous similar message [ 3441.133344] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3441.152644] Lustre: Skipped 2 previous similar messages [ 3446.241477] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3446.828844] Lustre: server umount lustre-MDT0000 complete [ 3450.885371] LustreError: 99234:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788802684 with bad export cookie 15710236109414261220 [ 3450.886355] LustreError: MGC192.168.202.121@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3450.892900] LustreError: 99234:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 3451.417895] Lustre: server umount lustre-MDT0001 complete [ 3464.477193] Lustre: DEBUG MARKER: oleg221-server.virtnet: executing unload_modules_local [ 3467.703231] Key type lgssc unregistered [ 3468.027366] LNet: 102789:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3468.040455] LNetError: 102789:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3468.088289] LNet: Removed LNI 192.168.202.121@tcp [ 3469.401458] Key type .llcrypt unregistered [ 3469.404479] Key type ._llcrypt unregistered [ 3487.900436] Key type ._llcrypt registered [ 3487.907387] Key type .llcrypt registered [ 3488.042988] Lustre: DEBUG MARKER: oleg221-server.virtnet: executing load_modules_local [ 3489.074298] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3489.139197] alg: No test for adler32 (adler32-zlib) [ 3490.542516] Lustre: Lustre: Build Version: 2.17.58_2_gefa5002 [ 3490.859363] LNet: Added LNI 192.168.202.121@tcp [8/256/0/180] [ 3492.663252] Key type lgssc registered [ 3494.010918] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3501.223563] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3501.239117] LustreError: 103871:0:(tgt_mount.c:2573:server_fill_super()) cfs_race id 716 sleeping [ 3501.245969] LustreError: 103872:0:(tgt_mount.c:2573:server_fill_super()) cfs_fail_race id 716 waking [ 3501.257220] LustreError: 103871:0:(tgt_mount.c:2573:server_fill_super()) cfs_fail_race id 716 awake: rc=4987 [ 3501.268427] LustreError: 103871:0:(tgt_mount.c:2585:server_fill_super()) Unable to start osd on /dev/mapper/mds1_flakey: -114 [ 3501.281763] LustreError: 103871:0:(super25.c:179:lustre_fill_super()) llite: Unable to mount : rc = -114 [ 3501.286843] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3503.147769] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3513.001259] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3513.398278] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3518.939688] Lustre: DEBUG MARKER: oleg221-server.virtnet: executing set_default_debug -1 all [ 3526.442216] Lustre: DEBUG MARKER: oleg221-server.virtnet: executing lsmod [ 3529.889079] LustreError: 105077:0:(tgt_mount.c:2573:server_fill_super()) cfs_race id 716 sleeping [ 3529.898540] LustreError: 105079:0:(tgt_mount.c:2573:server_fill_super()) cfs_fail_race id 716 waking [ 3529.902900] LustreError: 105077:0:(tgt_mount.c:2573:server_fill_super()) cfs_fail_race id 716 awake: rc=4994 [ 3529.913281] LustreError: 105077:0:(tgt_mount.c:2585:server_fill_super()) Unable to start osd on /dev/mapper/ost1_flakey: -114 [ 3529.922908] LustreError: 105077:0:(super25.c:179:lustre_fill_super()) llite: Unable to mount : rc = -114 [ 3529.928381] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3530.352206] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3533.795507] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3533.798955] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3533.835708] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3535.842114] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3535.843019] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3535.870434] Lustre: Skipped 1 previous similar message [ 3535.887133] Lustre: Skipped 1 previous similar message [ 3537.965101] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3537.990087] Lustre: Skipped 3 previous similar messages [ 3539.946097] Lustre: server umount lustre-MDT0000 complete [ 3543.011225] LustreError: 103905:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3543.036606] LustreError: 103905:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [ 3544.271187] LustreError: 103888:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788802777 with bad export cookie 16308409554292813017 [ 3544.279398] LustreError: MGC192.168.202.121@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3544.284714] LustreError: 103888:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 3546.207425] LustreError: 104341:0:(client.c:1394:ptlrpc_import_delay_req()) @@@ IMP_CLOSED req@000000004a13e030 x1875695605142272/t0(0) o900->lustre-MDT0000-lwp-MDT0001@0@lo:29/10 lens 264/248 e 0 to 0 dl 0 ref 2 fl Rpc:QU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 3546.433577] Lustre: server umount lustre-MDT0001 complete [ 3561.623314] Lustre: server umount lustre-OST0000 complete [ 3571.383047] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3571.796457] LustreError: 106143:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3571.831695] LustreError: 106143:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 3571.956295] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3577.083299] Lustre: DEBUG MARKER: oleg221-server.virtnet: executing set_default_debug -1 all [ 3577.331155] LustreError: 106144:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3582.437444] LustreError: 106143:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3587.556648] LustreError: 106144:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3587.651354] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3588.109972] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3594.701436] Lustre: DEBUG MARKER: oleg221-server.virtnet: executing set_default_debug -1 all [ 3604.539849] Lustre: DEBUG MARKER: oleg221-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3611.177623] Lustre: DEBUG MARKER: oleg221-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3619.757269] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3620.233192] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3627.610505] Lustre: DEBUG MARKER: oleg221-server.virtnet: executing set_default_debug -1 all [ 3637.170628] Lustre: DEBUG MARKER: oleg221-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3641.790444] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000400:3 to 0x280000400:65) [ 3652.065611] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3652.067624] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3652.073736] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3652.090121] Lustre: Skipped 1 previous similar message [ 3657.154497] Lustre: server umount lustre-OST0000 complete [ 3662.304504] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3662.308815] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3662.326228] Lustre: Skipped 1 previous similar message [ 3662.346168] Lustre: Skipped 2 previous similar messages [ 3667.852206] Lustre: server umount lustre-MDT0000 complete [ 3671.521669] LustreError: 106874:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3671.547792] LustreError: 106874:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 3671.911771] LustreError: 106123:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788802905 with bad export cookie 16308409554292813717 [ 3671.911771] LustreError: 106124:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788802905 with bad export cookie 16308409554292813717 [ 3671.914410] LustreError: MGC192.168.202.121@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3672.511681] Lustre: server umount lustre-MDT0001 complete [ 3683.362593] Lustre: DEBUG MARKER: oleg221-server.virtnet: executing unload_modules_local [ 3685.766540] Key type lgssc unregistered [ 3686.294892] LNet: 109698:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3686.316166] LNetError: 109698:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3686.342936] LNet: Removed LNI 192.168.202.121@tcp [ 3687.887404] Key type .llcrypt unregistered [ 3687.901235] Key type ._llcrypt unregistered [ 3724.291940] Key type ._llcrypt registered [ 3724.298321] Key type .llcrypt registered [ 3724.449359] Lustre: DEBUG MARKER: oleg221-server.virtnet: executing load_modules_local [ 3725.480439] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3725.515553] alg: No test for adler32 (adler32-zlib) [ 3726.681254] Lustre: Lustre: Build Version: 2.17.58_2_gefa5002 [ 3726.944930] LNet: Added LNI 192.168.202.121@tcp [8/256/0/180] [ 3728.663322] Key type lgssc registered [ 3730.090557] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3738.599555] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3738.627126] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3740.040075] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3744.848577] Lustre: DEBUG MARKER: oleg221-server.virtnet: executing set_default_debug -1 all [ 3745.257782] LustreError: 111115:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3745.279411] LustreError: 111115:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [ 3750.379288] LustreError: 111116:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3755.490185] LustreError: 111115:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3755.665720] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3756.163843] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3761.503304] Lustre: DEBUG MARKER: oleg221-server.virtnet: executing set_default_debug -1 all [ 3770.215903] Lustre: DEBUG MARKER: oleg221-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3777.159703] Lustre: DEBUG MARKER: oleg221-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3784.330671] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3784.884394] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3785.913699] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000400:67 to 0x280000400:97) [ 3792.889702] Lustre: DEBUG MARKER: oleg221-server.virtnet: executing set_default_debug -1 all [ 3802.863271] Lustre: DEBUG MARKER: oleg221-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3810.637765] Lustre: Setting parameter lustre-client.llite.some_wrong_param=10 in log lustre-client [ 3814.234097] Lustre: Modifying parameter lustre-client.llite.some_wrong_param=20 in log lustre-client [ 3822.050754] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3822.052448] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3822.082384] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3822.099989] Lustre: Skipped 1 previous similar message [ 3825.980782] Lustre: server umount lustre-OST0000 complete [ 3832.292898] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3832.319943] Lustre: Skipped 1 previous similar message [ 3832.351544] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3832.368143] Lustre: Skipped 1 previous similar message [ 3837.408952] LustreError: 112772:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3837.434691] LustreError: 112772:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 3838.912882] Lustre: server umount lustre-MDT0000 complete [ 3841.533900] LustreError: 111115:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3841.561984] LustreError: 111115:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 3845.114429] LustreError: 111095:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788803078 with bad export cookie 4790107694410133405 [ 3845.118608] LustreError: MGC192.168.202.121@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3845.125905] LustreError: 111095:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 3845.443965] Lustre: server umount lustre-MDT0001 complete [ 3860.472525] Lustre: DEBUG MARKER: oleg221-server.virtnet: executing unload_modules_local [ 3863.443766] Key type lgssc unregistered [ 3863.933437] LNet: 114755:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3863.946876] LNetError: 114755:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3863.968153] LNet: Removed LNI 192.168.202.121@tcp [ 3864.956751] Key type .llcrypt unregistered [ 3864.962329] Key type ._llcrypt unregistered [ 3892.125250] Key type ._llcrypt registered [ 3892.128619] Key type .llcrypt registered [ 3892.304480] Lustre: DEBUG MARKER: oleg221-server.virtnet: executing load_modules_local [ 3894.381682] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3894.464433] alg: No test for adler32 (adler32-zlib) [ 3895.561780] Lustre: Lustre: Build Version: 2.17.58_2_gefa5002 [ 3896.064441] LNet: Added LNI 192.168.202.121@tcp [8/256/0/180] [ 3897.815198] Key type lgssc registered [ 3899.705290] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3909.166514] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3909.185967] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3910.781192] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3915.833512] Lustre: DEBUG MARKER: oleg221-server.virtnet: executing set_default_debug -1 all [ 3921.377868] LustreError: 115976:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3921.389917] LustreError: 115976:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 3926.504300] LustreError: 115975:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3926.771070] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3927.448365] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3932.151301] Lustre: DEBUG MARKER: oleg221-server.virtnet: executing set_default_debug -1 all [ 3941.020488] Lustre: DEBUG MARKER: oleg221-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3948.924578] Lustre: DEBUG MARKER: oleg221-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3958.642034] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3959.047957] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3967.816525] Lustre: DEBUG MARKER: oleg221-server.virtnet: executing set_default_debug -1 all [ 3976.358199] Lustre: DEBUG MARKER: oleg221-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3980.546281] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000400:99 to 0x280000400:129) [ 3991.008386] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3991.017071] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3991.024560] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3991.062210] Lustre: Skipped 1 previous similar message [ 3996.132563] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3996.142759] Lustre: Skipped 2 previous similar messages [ 3996.876534] Lustre: server umount lustre-OST0000 complete [ 4004.324064] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4004.334844] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4004.362184] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4006.380143] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4006.385864] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4006.410836] Lustre: Skipped 1 previous similar message [ 4007.774510] Lustre: server umount lustre-MDT0000 complete [ 4011.488735] LustreError: 115976:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4011.517446] LustreError: 115976:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 4011.729691] LustreError: 115957:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788803245 with bad export cookie 4962807414596055220 [ 4011.758745] LustreError: MGC192.168.202.121@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4011.763456] LustreError: 115957:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 4012.111604] Lustre: server umount lustre-MDT0001 complete [ 4025.084582] Lustre: DEBUG MARKER: oleg221-server.virtnet: executing unload_modules_local [ 4029.104600] Key type lgssc unregistered [ 4029.540156] LNet: 119536:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4029.565334] LNetError: 119536:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4030.633903] LNet: Removed LNI 192.168.202.121@tcp [ 4031.859704] Key type .llcrypt unregistered [ 4031.864638] Key type ._llcrypt unregistered [ 4060.919529] Key type ._llcrypt registered [ 4060.921828] Key type .llcrypt registered [ 4061.006941] Lustre: DEBUG MARKER: oleg221-server.virtnet: executing load_modules_local [ 4062.636680] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4062.672231] alg: No test for adler32 (adler32-zlib) [ 4063.765020] Lustre: Lustre: Build Version: 2.17.58_2_gefa5002 [ 4064.063515] LNet: Added LNI 192.168.202.121@tcp [8/256/0/180] [ 4065.735196] Key type lgssc registered [ 4067.278179] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4075.224247] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4075.235684] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4076.730287] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4081.592805] Lustre: DEBUG MARKER: oleg221-server.virtnet: executing set_default_debug -1 all [ 4087.264947] LustreError: 120952:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4087.281624] LustreError: 120952:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 4091.254932] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4091.676703] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4096.905153] Lustre: DEBUG MARKER: oleg221-server.virtnet: executing set_default_debug -1 all [ 4107.247856] Lustre: DEBUG MARKER: oleg221-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4114.001803] Lustre: DEBUG MARKER: oleg221-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4122.491080] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4123.068272] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4129.719052] Lustre: DEBUG MARKER: oleg221-server.virtnet: executing set_default_debug -1 all [ 4139.194606] Lustre: DEBUG MARKER: oleg221-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4146.615361] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000400:131 to 0x280000400:161) [ 4155.747613] Lustre: Setting parameter lustre-MDT0000.mdt.root_squash=0:0 in log lustre-MDT0000 [ 4159.582609] Lustre: Setting parameter lustre-MDT0000.mdt.nosquash_nids=NONE in log lustre-MDT0000 [ 4159.591140] Lustre: Skipped 2 previous similar messages [ 4162.216785] Lustre: lustre-MDT0001: root_squash is set to 0:0 [ 4162.281070] Lustre: lustre-MDT0001: nosquash_nids is cleared [ 4164.402165] Lustre: Modifying parameter lustre-MDT0000.mdt.root_squash=500:500 in log lustre-MDT0000 [ 4164.409873] Lustre: Skipped 2 previous similar messages [ 4167.888325] Lustre: lustre-MDT0001: root_squash is set to 500:500 [ 4167.897301] Lustre: Skipped 1 previous similar message [ 4178.956182] Lustre: Modifying parameter lustre-MDT0000.mdt.nosquash_nids=2@gni 0@lo 192.168.202.21@tcp 192.168.0.[2,10]@tcp in log lustre-MDT0000 [ 4178.964739] Lustre: Skipped 2 previous similar messages [ 4185.248673] Lustre: lustre-MDT0001: nosquash_nids set to 2@gni 0@lo 192.168.202.21@tcp 192.168.0.[2,10]@tcp [ 4191.820368] Lustre: Modifying parameter lustre-MDT0000.mdt.nosquash_nids=192.168.202.21/24@tcp in log lustre-MDT0000 [ 4191.826086] Lustre: Skipped 2 previous similar messages [ 4193.958736] Lustre: lustre-MDT0001: nosquash_nids set to 192.168.202.21/24@tcp [ 4193.971138] Lustre: Skipped 1 previous similar message [ 4200.936502] LustreError: lustre-OST0000-osc-MDT0000: operation ost_destroy to node 0@lo failed: rc = -107 [ 4200.938786] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4200.947568] LustreError: Skipped 2 previous similar messages [ 4200.979727] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4202.984646] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4202.985817] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4206.973458] Lustre: server umount lustre-OST0000 complete [ 4213.217909] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4213.234473] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4213.245908] Lustre: Skipped 2 previous similar messages [ 4217.594712] Lustre: server umount lustre-MDT0000 complete [ 4218.336178] LustreError: 121937:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4218.362823] LustreError: 121937:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 4221.561030] LustreError: 123603:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788803455 with bad export cookie 987509172577627875 [ 4221.562599] LustreError: MGC192.168.202.121@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4221.571460] LustreError: 123603:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4222.063614] Lustre: server umount lustre-MDT0001 complete [ 4234.710759] Lustre: DEBUG MARKER: oleg221-server.virtnet: executing unload_modules_local [ 4237.547389] Key type lgssc unregistered [ 4237.927159] LNet: 125822:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4237.938379] LNetError: 125822:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4237.952264] LNet: Removed LNI 192.168.202.121@tcp [ 4238.902177] Key type .llcrypt unregistered [ 4238.906362] Key type ._llcrypt unregistered [ 4268.230498] Key type ._llcrypt registered [ 4268.233705] Key type .llcrypt registered [ 4268.371480] Lustre: DEBUG MARKER: oleg221-server.virtnet: executing load_modules_local [ 4269.502458] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4269.668984] alg: No test for adler32 (adler32-zlib) [ 4270.732145] Lustre: Lustre: Build Version: 2.17.58_2_gefa5002 [ 4270.956128] LNet: Added LNI 192.168.202.121@tcp [8/256/0/180] [ 4272.599264] Key type lgssc registered [ 4273.856139] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4281.507574] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4281.526596] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4283.157745] Lustre: lustre-MDT0000: root_squash is set to 500:500 [ 4283.194836] Lustre: lustre-MDT0000: nosquash_nids set to 192.168.202.21/24@tcp [ 4283.229978] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4287.924819] Lustre: DEBUG MARKER: oleg221-server.virtnet: executing set_default_debug -1 all [ 4292.579629] LustreError: 127382:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4292.607997] LustreError: 127382:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 4297.726895] LustreError: 127383:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4298.006597] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4298.323875] Lustre: lustre-MDT0001: root_squash is set to 500:500 [ 4298.358723] Lustre: lustre-MDT0001: nosquash_nids set to 192.168.202.21/24@tcp [ 4298.412018] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4303.794227] Lustre: DEBUG MARKER: oleg221-server.virtnet: executing set_default_debug -1 all [ 4312.196956] Lustre: DEBUG MARKER: oleg221-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4318.545413] Lustre: DEBUG MARKER: oleg221-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4326.582449] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4326.912099] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4327.945731] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000400:169 to 0x280000400:193) [ 4335.048628] Lustre: DEBUG MARKER: oleg221-server.virtnet: executing set_default_debug -1 all [ 4343.613890] Lustre: DEBUG MARKER: oleg221-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4358.637777] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4358.641305] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4358.651424] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4358.659330] Lustre: Skipped 1 previous similar message [ 4358.682865] LustreError: Skipped 1 previous similar message [ 4360.293398] Lustre: server umount lustre-OST0000 complete [ 4364.769150] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4364.781431] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4364.798286] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4364.803778] Lustre: Skipped 1 previous similar message [ 4368.866548] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4368.871095] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4368.885530] Lustre: Skipped 1 previous similar message [ 4370.385756] Lustre: server umount lustre-MDT0000 complete [ 4373.986799] LustreError: 127379:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4374.016272] LustreError: 127379:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 4374.755125] LustreError: 127363:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788803608 with bad export cookie 10869821683954561189 [ 4374.758934] LustreError: MGC192.168.202.121@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4374.788461] LustreError: 127363:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4375.139734] Lustre: server umount lustre-MDT0001 complete [ 4387.042441] Lustre: DEBUG MARKER: oleg221-server.virtnet: executing unload_modules_local [ 4390.623954] Key type lgssc unregistered [ 4390.968605] LNet: 130971:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4390.974438] LNetError: 130971:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4391.000359] LNet: Removed LNI 192.168.202.121@tcp [ 4392.077889] Key type .llcrypt unregistered [ 4392.080211] Key type ._llcrypt unregistered [ 4419.633254] Key type ._llcrypt registered [ 4419.636390] Key type .llcrypt registered [ 4419.748770] Lustre: DEBUG MARKER: oleg221-server.virtnet: executing set_hostid [ 4435.678297] Lustre: DEBUG MARKER: oleg221-server.virtnet: executing load_modules_local [ 4437.086179] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4437.185100] alg: No test for adler32 (adler32-zlib) [ 4438.344505] Lustre: Lustre: Build Version: 2.17.58_2_gefa5002 [ 4438.585669] LNet: Added LNI 192.168.202.121@tcp [8/256/0/180] [ 4440.336314] Key type lgssc registered [ 4441.627779] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4480.006183] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4480.030693] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4481.283324] Lustre: First target registration detected: opening a one-time 24-hour registration window. Use 'lctl set_param allow_register=0' to close it early, or 'lctl set_param -P allow_register=0' to lock down permanently. [ 4481.302831] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 4481.323750] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 4481.383129] Lustre: lustre-MDT0000: new disk, initializing [ 4481.448441] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4481.478196] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 4486.679039] Lustre: DEBUG MARKER: oleg221-server.virtnet: executing set_default_debug -1 all [ 4499.553134] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4499.751874] Lustre: 134372:0:(mgs_llog.c:1462:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 4499.843531] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 4499.849209] Lustre: Skipped 1 previous similar message [ 4500.009229] Lustre: lustre-MDT0001: new disk, initializing [ 4500.185291] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4500.222980] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 4500.234890] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 4506.667219] Lustre: DEBUG MARKER: oleg221-server.virtnet: executing set_default_debug -1 all [ 4518.071269] Lustre: DEBUG MARKER: oleg221-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4525.283172] Lustre: DEBUG MARKER: oleg221-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4533.879191] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4534.360068] Lustre: lustre-OST0000: new disk, initializing [ 4534.369754] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 4534.379852] Lustre: 136338:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 4534.506046] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4539.968949] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 4539.980710] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 4540.100406] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 4541.713465] Lustre: DEBUG MARKER: oleg221-server.virtnet: executing set_default_debug -1 all [ 4554.528157] Lustre: DEBUG MARKER: oleg221-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4560.737772] Lustre: DEBUG MARKER: oleg221-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 4560.995224] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 4565.424500] Lustre: DEBUG MARKER: oleg221-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 4565.694560] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 4571.110762] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4571.131201] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4571.157555] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4571.164406] Lustre: Skipped 1 previous similar message [ 4576.235557] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4581.346120] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4581.355459] Lustre: Skipped 2 previous similar messages [ 4582.367425] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 4582.571022] Lustre: server umount lustre-OST0000 complete [ 4587.488771] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4587.493447] LustreError: Skipped 1 previous similar message [ 4587.495887] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4587.509484] Lustre: Skipped 1 previous similar message [ 4587.513536] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4591.586503] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4591.594563] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4591.620372] Lustre: Skipped 1 previous similar message [ 4593.186394] Lustre: server umount lustre-MDT0000 complete [ 4597.196684] LustreError: 134363:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788803830 with bad export cookie 10939137782931303242 [ 4597.203845] LustreError: MGC192.168.202.121@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4597.209500] LustreError: 134363:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 4597.607946] Lustre: server umount lustre-MDT0001 complete [ 4607.542424] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4608.065446] LustreError: 138558:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4608.094860] LustreError: 138558:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [ 4608.183099] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4613.610416] LustreError: 138559:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4613.653747] Lustre: DEBUG MARKER: oleg221-server.virtnet: executing set_default_debug -1 all [ 4617.704080] LustreError: 138558:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4622.819832] LustreError: 138559:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4623.429911] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4623.897968] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4629.725816] Lustre: DEBUG MARKER: oleg221-server.virtnet: executing set_default_debug -1 all [ 4638.397314] Lustre: DEBUG MARKER: oleg221-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4645.571296] Lustre: DEBUG MARKER: oleg221-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4653.027715] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4653.482381] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4661.135874] Lustre: DEBUG MARKER: oleg221-server.virtnet: executing set_default_debug -1 all [ 4669.748548] Lustre: DEBUG MARKER: oleg221-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4675.159359] Lustre: DEBUG MARKER: oleg221-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 4675.363267] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 4681.184990] Lustre: DEBUG MARKER: oleg221-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 4681.443819] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 4691.212229] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4691.387400] Lustre: lustre-OST0001: new disk, initializing [ 4691.391438] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 4691.397281] Lustre: 141930:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 4691.498707] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 4699.270592] Lustre: DEBUG MARKER: oleg221-server.virtnet: executing set_default_debug -1 all [ 4700.339552] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 4700.358477] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 4700.512489] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 4708.491668] Lustre: DEBUG MARKER: oleg221-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid 50 [ 4708.668481] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 4714.409686] Lustre: DEBUG MARKER: oleg221-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [ 4714.665186] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 4721.554909] Lustre: DEBUG MARKER: oleg221-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff975b87010800.ost_server_uuid 50 [ 4723.128908] Lustre: DEBUG MARKER: osc.lustre-OST0001-osc-ffff975b87010800.ost_server_uuid in IDLE state after 0 sec [ 4736.483192] Lustre: lustre-OST0001-osc-MDT0001: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4736.491913] LustreError: lustre-OST0001-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4736.533616] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 4746.207247] Lustre: lustre-OST0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 4746.288853] Lustre: server umount lustre-OST0001 complete [ 4746.727516] LustreError: 140381:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4746.739801] LustreError: 140381:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 4750.816854] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4750.820455] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4750.835361] Lustre: Skipped 1 previous similar message [ 4756.381582] Lustre: server umount lustre-OST0000 complete [ 4762.081708] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4762.097720] Lustre: Skipped 1 previous similar message [ 4762.117929] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4762.120312] Lustre: Skipped 6 previous similar messages [ 4766.918031] Lustre: server umount lustre-MDT0000 complete [ 4767.201115] LustreError: 139487:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4767.213906] LustreError: 139487:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [ 4770.792896] LustreError: 138539:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788804004 with bad export cookie 10939137782931304124 [ 4770.802831] LustreError: MGC192.168.202.121@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4770.819445] LustreError: 138539:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 5 previous similar messages [ 4771.066528] Lustre: server umount lustre-MDT0001 complete [ 4794.274585] Lustre: DEBUG MARKER: oleg221-server.virtnet: executing unload_modules_local [ 4797.273991] Key type lgssc unregistered [ 4797.516604] LNet: 144935:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4797.539063] LNetError: 144935:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4797.555951] LNet: Removed LNI 192.168.202.121@tcp [ 4798.625624] Key type .llcrypt unregistered [ 4798.628225] Key type ._llcrypt unregistered [ 4846.870610] Key type ._llcrypt registered [ 4846.872207] Key type .llcrypt registered [ 4847.060489] Lustre: DEBUG MARKER: oleg221-server.virtnet: executing set_hostid [ 4867.705482] Lustre: DEBUG MARKER: oleg221-server.virtnet: executing load_modules_local [ 4869.350118] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4869.394819] alg: No test for adler32 (adler32-zlib) [ 4870.500959] Lustre: Lustre: Build Version: 2.17.58_2_gefa5002 [ 4870.738549] LNet: Added LNI 192.168.202.121@tcp [8/256/0/180] [ 4872.463318] Key type lgssc registered [ 4874.013748] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4912.900041] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4912.927236] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4914.157900] Lustre: First target registration detected: opening a one-time 24-hour registration window. Use 'lctl set_param allow_register=0' to close it early, or 'lctl set_param -P allow_register=0' to lock down permanently. [ 4914.195965] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 4914.236182] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 4914.383463] Lustre: lustre-MDT0000: new disk, initializing [ 4914.484900] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4914.505267] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 4919.345594] Lustre: DEBUG MARKER: oleg221-server.virtnet: executing set_default_debug -1 all [ 4930.830830] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4930.948394] Lustre: 149103:0:(mgs_llog.c:1462:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 4930.999786] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 4931.004068] Lustre: Skipped 1 previous similar message [ 4931.107825] Lustre: lustre-MDT0001: new disk, initializing [ 4931.240492] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4931.305296] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 4931.320076] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 4936.060452] Lustre: DEBUG MARKER: oleg221-server.virtnet: executing set_default_debug -1 all [ 4946.533209] Lustre: DEBUG MARKER: oleg221-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4952.602256] Lustre: DEBUG MARKER: oleg221-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4961.428094] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4961.718156] Lustre: lustre-OST0000: new disk, initializing [ 4961.721370] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 4961.726536] Lustre: 151071:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 4961.810618] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4967.511239] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 4967.530531] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 4967.673085] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 4968.697729] Lustre: DEBUG MARKER: oleg221-server.virtnet: executing set_default_debug -1 all [ 4979.579092] Lustre: DEBUG MARKER: oleg221-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4985.118921] Lustre: Failing over lustre-OST0000 [ 4986.851893] LustreError: lustre-OST0000-osc-MDT0000: operation ost_destroy to node 0@lo failed: rc = -107 [ 4986.865791] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4986.915066] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4987.479969] Lustre: server umount lustre-OST0000 complete [ 5004.258275] Lustre: 147323:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1788804221/real 1788804221] req@00000000087b73b9 x1875697052192128/t0(0) o13->lustre-OST0000-osc-MDT0001@0@lo:7/4 lens 224/368 e 0 to 1 dl 1788804237 ref 1 fl Rpc:XQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 5004.305572] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5006.599735] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5006.895475] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5006.920668] Lustre: lustre-OST0000: in recovery but waiting for the first client to connect [ 5010.383227] Lustre: 147324:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1788804227/real 1788804227] req@00000000be10a928 x1875697052193920/t0(0) o400->lustre-OST0000-osc-MDT0001@0@lo:28/4 lens 224/224 e 0 to 1 dl 1788804243 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 5014.060102] Lustre: lustre-OST0000: Will be in recovery for at least 1:00, or until 3 clients reconnect [ 5014.121331] Lustre: DEBUG MARKER: oleg221-server.virtnet: executing set_default_debug -1 all [ 5015.072407] Lustre: 147324:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1788804232/real 1788804232] req@0000000020adae7c x1875697052194816/t0(0) o400->lustre-OST0000-osc-MDT0001@0@lo:28/4 lens 224/224 e 0 to 1 dl 1788804248 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 5018.667568] Lustre: Failing over lustre-MDT0000 [ 5018.873373] Lustre: server umount lustre-MDT0000 complete [ 5019.109526] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5019.129197] Lustre: Skipped 1 previous similar message [ 5019.680170] Lustre: 147325:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1788804236/real 1788804236] req@000000004613a928 x1875697052195712/t0(0) o400->lustre-OST0000-osc-MDT0001@0@lo:28/4 lens 224/224 e 0 to 1 dl 1788804252 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 5024.225658] LustreError: 149108:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5024.241494] LustreError: 149108:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 7 previous similar messages [ 5029.344533] LustreError: 150227:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5029.365542] LustreError: 150227:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [ 5034.465793] LustreError: 149114:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5034.493225] LustreError: 149114:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [ 5035.363337] Lustre: 147326:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1788804252/real 1788804252] req@00000000c91e0c7b x1875697052205184/t0(0) o400->MGC192.168.202.121@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1788804268 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 5035.409394] LustreError: MGC192.168.202.121@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5037.817065] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5040.609956] LustreError: 149114:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5040.645713] LustreError: 149114:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 7 previous similar messages [ 5044.713457] LustreError: 149113:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5044.743242] LustreError: 149113:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [ 5044.985930] LustreError: lustre-OST0000-osc-MDT0000: operation ost_connect to node 0@lo failed: rc = -114 [ 5045.041672] Lustre: lustre-OST0000: Recovery over after 0:31, of 3 clients 3 recovered and 0 were evicted. [ 5045.041779] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5045.047134] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5045.122420] Lustre: lustre-MDT0000: in recovery but waiting for the first client to connect [ 5045.831948] Lustre: lustre-MDT0000: Will be in recovery for at least 1:00, or until 2 clients reconnect [ 5049.457629] Lustre: DEBUG MARKER: oleg221-server.virtnet: executing set_default_debug -1 all [ 5050.363296] Lustre: lustre-MDT0000-lwp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 5050.372296] Lustre: Skipped 1 previous similar message [ 5050.386076] Lustre: lustre-MDT0000: Recovery over after 0:05, of 2 clients 2 recovered and 0 were evicted. [ 5050.441462] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:3 to 0x280000401:33) [ 5060.580828] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5060.591306] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5060.601388] Lustre: Skipped 2 previous similar messages [ 5060.603184] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 5060.608463] Lustre: Skipped 1 previous similar message [ 5063.586428] Lustre: server umount lustre-OST0000 complete [ 5070.825383] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5070.834237] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 5070.840400] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5070.857200] Lustre: Skipped 1 previous similar message [ 5074.678119] Lustre: server umount lustre-MDT0000 complete [ 5075.936396] LustreError: 149114:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5075.953101] LustreError: 149114:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 5078.477517] LustreError: 149093:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788804312 with bad export cookie 10527156717609064789 [ 5078.499635] LustreError: MGC192.168.202.121@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5078.835578] Lustre: server umount lustre-MDT0001 complete [ 5091.145631] Lustre: DEBUG MARKER: oleg221-server.virtnet: executing unload_modules_local [ 5094.153916] Key type lgssc unregistered [ 5094.443753] LNet: 154725:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5094.454627] LNetError: 154725:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5095.531081] LNet: Removed LNI 192.168.202.121@tcp [ 5096.508185] Key type .llcrypt unregistered [ 5096.511211] Key type ._llcrypt unregistered [ 5138.454307] Key type ._llcrypt registered [ 5138.456617] Key type .llcrypt registered [ 5138.607402] Lustre: DEBUG MARKER: oleg221-server.virtnet: executing load_modules_local [ 5139.847418] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5139.937843] alg: No test for adler32 (adler32-zlib) [ 5140.975838] Lustre: Lustre: Build Version: 2.17.58_2_gefa5002 [ 5141.173603] LNet: Added LNI 192.168.202.121@tcp [8/256/0/180] [ 5142.815480] Key type lgssc registered [ 5144.015795] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5151.939562] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5151.959342] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5153.469059] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5158.479217] Lustre: DEBUG MARKER: oleg221-server.virtnet: executing set_default_debug -1 all [ 5162.987232] LustreError: 156242:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5163.007044] LustreError: 156242:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 5167.995187] Lustre: DEBUG MARKER: oleg221-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 5168.100962] LustreError: 156243:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5173.114222] LustreError: 156271:0:(osp_object.c:618:osp_attr_get()) lustre-MDT0001-osp-MDT0000: osp_attr_get update error [0x200000009:0x1:0x0]: rc = -5 [ 5173.145835] LustreError: 156271:0:(lod_sub_object.c:917:lod_sub_prep_llog()) lustre-MDT0000-mdtlov: can't get id from catalogs: rc = -5 [ 5173.163536] LustreError: 156271:0:(lod_dev.c:511:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 19, retries 0, failed: rc = -5 [ 5173.600243] Lustre: server umount lustre-MDT0000 complete [ 5193.374425] Lustre: DEBUG MARKER: oleg221-server.virtnet: executing set_hostid [ 5207.352645] Lustre: DEBUG MARKER: oleg221-server.virtnet: executing load_modules_local [ 5248.622672] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5248.849453] Lustre: First target registration detected: opening a one-time 24-hour registration window. Use 'lctl set_param allow_register=0' to close it early, or 'lctl set_param -P allow_register=0' to lock down permanently. [ 5248.909977] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 5248.967533] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 5249.064929] Lustre: lustre-MDT0000: new disk, initializing [ 5249.147417] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5249.162572] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 5253.620568] Lustre: DEBUG MARKER: oleg221-server.virtnet: executing set_default_debug -1 all [ 5265.390765] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5265.499618] Lustre: 159596:0:(mgs_llog.c:1462:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 5265.553158] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 5265.557765] Lustre: Skipped 1 previous similar message [ 5265.627952] Lustre: lustre-MDT0001: new disk, initializing [ 5265.720846] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5265.758692] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 5265.764227] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 5270.890745] Lustre: DEBUG MARKER: oleg221-server.virtnet: executing set_default_debug -1 all [ 5283.798465] Lustre: DEBUG MARKER: oleg221-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 5292.964096] Lustre: DEBUG MARKER: oleg221-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 5300.834382] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5301.146267] Lustre: lustre-OST0000: new disk, initializing [ 5301.150750] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 5301.158531] Lustre: 161567:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 5301.236961] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5308.223403] Lustre: DEBUG MARKER: oleg221-server.virtnet: executing set_default_debug -1 all [ 5310.038653] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 5310.053054] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 5310.192384] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 5319.247869] Lustre: DEBUG MARKER: oleg221-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 5324.535871] Lustre: DEBUG MARKER: SKIP: conf-sanity test_47b Can not create symlink /usr/sbin/lctl [ 5326.802607] Lustre: DEBUG MARKER: == conf-sanity test 48: too many acls on file ============ 14:09:19 (1788804559) [ 5608.928738] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 5608.942048] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5608.958285] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5612.515531] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5612.527378] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5612.541133] Lustre: Skipped 1 previous similar message [ 5616.600826] Lustre: server umount lustre-MDT0000 complete [ 5617.633416] LustreError: 159610:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5617.662030] LustreError: 159610:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 5620.998420] LustreError: 159590:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788804854 with bad export cookie 6173786289570447418 [ 5621.005421] LustreError: MGC192.168.202.121@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5621.014543] LustreError: 159590:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 5622.752246] LustreError: 159605:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5622.752838] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5622.788193] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 5622.795399] Lustre: Skipped 2 previous similar messages [ 5625.762248] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 5627.616496] Lustre: server umount lustre-MDT0001 complete [ 5638.651188] Lustre: server umount lustre-OST0000 complete [ 5651.357854] Lustre: DEBUG MARKER: oleg221-server.virtnet: executing set_hostid [ 5665.577562] Lustre: DEBUG MARKER: oleg221-server.virtnet: executing load_modules_local [ 5705.287209] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5705.663141] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 5705.726195] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 5705.871888] Lustre: lustre-MDT0000: new disk, initializing [ 5706.030312] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5706.077899] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 5711.016848] Lustre: DEBUG MARKER: oleg221-server.virtnet: executing set_default_debug -1 all [ 5725.608641] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5725.825453] Lustre: 166236:0:(mgs_llog.c:1462:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 5725.957602] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 5725.966777] Lustre: Skipped 1 previous similar message [ 5726.110558] Lustre: lustre-MDT0001: new disk, initializing [ 5726.276430] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5726.351519] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 5726.364714] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 5731.642656] Lustre: DEBUG MARKER: oleg221-server.virtnet: executing set_default_debug -1 all [ 5741.352470] Lustre: DEBUG MARKER: oleg221-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 5747.974219] Lustre: DEBUG MARKER: oleg221-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 5755.433070] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5755.734495] Lustre: lustre-OST0000: new disk, initializing [ 5755.738592] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 5755.748395] Lustre: 168205:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 5755.826381] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5762.825209] Lustre: DEBUG MARKER: oleg221-server.virtnet: executing set_default_debug -1 all [ 5765.224591] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 5765.243745] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 5765.364304] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 5774.517734] Lustre: DEBUG MARKER: oleg221-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 5779.911631] Lustre: DEBUG MARKER: oleg221-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 5780.215827] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 5785.505207] Lustre: DEBUG MARKER: oleg221-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 5785.790230] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 5791.221855] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 5791.224188] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5791.231502] LustreError: Skipped 1 previous similar message [ 5791.238361] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5791.266357] Lustre: Skipped 1 previous similar message [ 5794.102573] Lustre: server umount lustre-OST0000 complete [ 5801.441754] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5801.450484] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5801.464789] Lustre: Skipped 1 previous similar message [ 5801.482962] Lustre: Skipped 2 previous similar messages [ 5804.830305] Lustre: server umount lustre-MDT0000 complete [ 5806.562906] LustreError: 166247:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5806.603490] LustreError: 166247:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 5808.823237] LustreError: 166227:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788805042 with bad export cookie 6173786289570606255 [ 5808.828433] LustreError: MGC192.168.202.121@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5808.830855] LustreError: 166227:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 5809.139174] Lustre: server umount lustre-MDT0001 complete [ 5818.539043] Lustre: DEBUG MARKER: == conf-sanity test 49a: check PARAM_SYS_LDLM_TIMEOUT option of mkfs.lustre ========================================================== 14:17:30 (1788805050) [ 5837.893907] Lustre: DEBUG MARKER: oleg221-server.virtnet: executing set_hostid [ 5849.958288] Lustre: DEBUG MARKER: oleg221-server.virtnet: executing load_modules_local [ 5888.853644] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5889.099811] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 5889.141164] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 5889.334948] Lustre: lustre-MDT0000: new disk, initializing [ 5889.468991] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5889.503833] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 5894.946503] Lustre: DEBUG MARKER: oleg221-server.virtnet: executing set_default_debug -1 all [ 5906.602930] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5906.760844] Lustre: 172960:0:(mgs_llog.c:1462:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 5907.044431] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 5912.339270] Lustre: DEBUG MARKER: oleg221-server.virtnet: executing set_default_debug -1 all [ 5922.943469] Lustre: DEBUG MARKER: oleg221-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 5930.621227] Lustre: DEBUG MARKER: oleg221-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 5937.587042] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5937.913877] Lustre: lustre-OST0000: new disk, initializing [ 5937.916910] Lustre: Skipped 1 previous similar message [ 5937.920167] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 5937.922450] Lustre: Skipped 2 previous similar messages [ 5937.927975] Lustre: 174939:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 5944.420021] Lustre: DEBUG MARKER: oleg221-server.virtnet: executing set_default_debug -1 all [ 5948.055403] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 5948.064807] Lustre: Skipped 1 previous similar message [ 5948.069953] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 5948.280798] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 5954.977525] Lustre: DEBUG MARKER: oleg221-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 5963.745772] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 5963.750912] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5963.753436] LustreError: Skipped 1 previous similar message [ 5963.781453] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5968.272640] Lustre: server umount lustre-OST0000 complete [ 5973.479646] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 5978.563029] Lustre: server umount lustre-MDT0000 complete [ 5979.108713] LustreError: 175610:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5979.127157] LustreError: 175610:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 5982.781780] LustreError: 172953:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788805216 with bad export cookie 6173786289570607137 [ 5982.800778] LustreError: MGC192.168.202.121@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5982.805282] LustreError: 172953:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 5983.496382] Lustre: server umount lustre-MDT0001 complete [ 5994.055903] Lustre: DEBUG MARKER: == conf-sanity test 49b: check PARAM_SYS_LDLM_TIMEOUT option of mkfs.lustre ========================================================== 14:20:26 (1788805226) [ 6012.873562] Lustre: DEBUG MARKER: oleg221-server.virtnet: executing set_hostid [ 6025.398701] Lustre: DEBUG MARKER: oleg221-server.virtnet: executing load_modules_local [ 6065.527603] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6065.849846] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 6065.896035] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 6066.005678] Lustre: lustre-MDT0000: new disk, initializing [ 6066.076762] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6066.084615] Lustre: Skipped 2 previous similar messages [ 6066.102290] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 6071.286817] Lustre: DEBUG MARKER: oleg221-server.virtnet: executing set_default_debug -1 all [ 6082.922132] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6083.044806] Lustre: 179266:0:(mgs_llog.c:1462:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 6083.391061] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 6089.021305] Lustre: DEBUG MARKER: oleg221-server.virtnet: executing set_default_debug -1 all [ 6099.227232] Lustre: DEBUG MARKER: oleg221-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 6105.223983] Lustre: DEBUG MARKER: oleg221-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 6113.640969] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6113.999776] Lustre: 181245:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 6121.065654] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 6121.206594] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 6122.005656] Lustre: DEBUG MARKER: oleg221-server.virtnet: executing set_default_debug -1 all [ 6132.714598] Lustre: DEBUG MARKER: oleg221-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 6141.922257] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 6141.932550] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6141.957942] Lustre: Skipped 3 previous similar messages [ 6141.965484] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6141.973104] Lustre: Skipped 3 previous similar messages [ 6154.210339] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 6154.380831] Lustre: server umount lustre-OST0000 complete [ 6160.354971] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 6160.369561] LustreError: Skipped 1 previous similar message [ 6165.248914] Lustre: server umount lustre-MDT0000 complete [ 6167.520515] LustreError: 179278:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6167.555282] LustreError: 179278:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 6169.234794] LustreError: 179258:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788805402 with bad export cookie 6173786289570608229 [ 6169.236555] LustreError: MGC192.168.202.121@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6169.245945] LustreError: 179258:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 6169.594605] Lustre: server umount lustre-MDT0001 complete [ 6182.536858] Lustre: DEBUG MARKER: oleg221-server.virtnet: executing unload_modules_local [ 6186.641770] Key type lgssc unregistered [ 6187.089711] LNet: 183102:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 6187.099246] LNetError: 183102:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 6187.129557] LNet: Removed LNI 192.168.202.121@tcp [ 6188.204912] Key type .llcrypt unregistered [ 6188.206937] Key type ._llcrypt unregistered [ 6214.042746] Key type ._llcrypt registered [ 6214.046148] Key type .llcrypt registered [ 6214.168934] Lustre: DEBUG MARKER: oleg221-server.virtnet: executing load_modules_local [ 6215.336440] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 6215.435586] alg: No test for adler32 (adler32-zlib) [ 6216.601483] Lustre: Lustre: Build Version: 2.17.58_2_gefa5002 [ 6216.876100] LNet: Added LNI 192.168.202.121@tcp [8/256/0/180] [ 6218.575390] Key type lgssc registered [ 6219.549091] Lustre: Echo OBD driver; http://www.lustre.org/ [ 6227.109760] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 6227.142355] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6228.617937] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6232.986739] Lustre: DEBUG MARKER: oleg221-server.virtnet: executing set_default_debug -1 all [ 6239.221794] LustreError: 184515:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6239.247249] LustreError: 184515:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 6241.145913] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6241.545596] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 6246.373710] Lustre: DEBUG MARKER: oleg221-server.virtnet: executing set_default_debug -1 all [ 6254.696685] Lustre: DEBUG MARKER: oleg221-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 6260.615044] Lustre: DEBUG MARKER: oleg221-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 6267.867448] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6268.295103] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 6274.487182] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:3 to 0x280000401:33) [ 6274.491928] Lustre: DEBUG MARKER: oleg221-server.virtnet: executing set_default_debug -1 all [ 6282.461256] Lustre: DEBUG MARKER: oleg221-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 6305.248868] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 6305.257931] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6305.275546] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6306.274078] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6306.278487] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 6306.287849] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6311.072061] Lustre: server umount lustre-OST0000 complete [ 6316.524173] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6316.547704] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6316.549584] Lustre: Skipped 1 previous similar message [ 6316.569506] Lustre: Skipped 2 previous similar messages [ 6321.634756] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6321.642975] Lustre: Skipped 1 previous similar message [ 6325.731449] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6325.734915] Lustre: Skipped 1 previous similar message [ 6329.311519] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 6329.547885] Lustre: server umount lustre-MDT0000 complete [ 6330.852279] LustreError: 184508:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6330.872829] LustreError: 184508:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 6333.376309] LustreError: 185248:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788805566 with bad export cookie 17900478547812923331 [ 6333.383659] LustreError: MGC192.168.202.121@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6333.386241] LustreError: 185248:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 6333.872774] Lustre: server umount lustre-MDT0001 complete [ 6346.774240] Lustre: DEBUG MARKER: oleg221-server.virtnet: executing unload_modules_local [ 6350.050396] Key type lgssc unregistered [ 6350.363145] LNet: 188073:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 6350.374502] LNetError: 188073:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 6350.393347] LNet: Removed LNI 192.168.202.121@tcp [ 6351.186937] Key type .llcrypt unregistered [ 6351.190065] Key type ._llcrypt unregistered [ 6375.607415] Key type ._llcrypt registered [ 6375.610445] Key type .llcrypt registered [ 6375.699152] Lustre: DEBUG MARKER: oleg221-server.virtnet: executing load_modules_local [ 6376.332223] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 6376.347348] alg: No test for adler32 (adler32-zlib) [ 6377.592196] Lustre: Lustre: Build Version: 2.17.58_2_gefa5002 [ 6377.860771] LNet: Added LNI 192.168.202.121@tcp [8/256/0/180] [ 6379.577691] Key type lgssc registered [ 6380.677318] Lustre: Echo OBD driver; http://www.lustre.org/ [ 6388.151882] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 6388.171763] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6389.528559] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6393.202247] Lustre: DEBUG MARKER: oleg221-server.virtnet: executing set_default_debug -1 all [ 6399.972032] LustreError: 189484:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6399.992084] LustreError: 189484:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 6402.783150] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6403.136309] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 6407.623861] Lustre: DEBUG MARKER: oleg221-server.virtnet: executing set_default_debug -1 all [ 6415.478573] Lustre: DEBUG MARKER: oleg221-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 6421.615180] Lustre: DEBUG MARKER: oleg221-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 6428.259258] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6428.588697] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 6434.745620] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:35 to 0x280000401:65) [ 6435.208943] Lustre: DEBUG MARKER: oleg221-server.virtnet: executing set_default_debug -1 all [ 6443.813906] Lustre: DEBUG MARKER: oleg221-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 6455.266822] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 6455.275454] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6455.289271] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6456.923693] Lustre: lustre-OST0000: Not available for connect from 192.168.202.21@tcp (stopping) [ 6458.339813] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6458.343066] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6460.115351] Lustre: server umount lustre-OST0000 complete [ 6468.534694] Lustre: DEBUG MARKER: oleg221-client.virtnet: executing wait_import_state DISCONN osc.lustre-OST0000-osc-ffff975b84b90000.ost_server_uuid 50 [ 6470.754733] Lustre: DEBUG MARKER: OSCs should all be DISCONN [ 6489.061230] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6489.073146] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6489.091225] Lustre: Skipped 2 previous similar messages [ 6491.507245] Lustre: server umount lustre-MDT0000 complete [ 6494.176987] LustreError: 189481:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6494.200973] LustreError: 189481:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 6495.086944] LustreError: 189466:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788805728 with bad export cookie 3348492360261672866 [ 6495.095936] LustreError: 189466:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 6495.097012] LustreError: MGC192.168.202.121@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6495.429815] Lustre: server umount lustre-MDT0001 complete [ 6504.962096] Lustre: DEBUG MARKER: == conf-sanity test 50c: lazystatfs one server down ====== 14:28:57 (1788805737) [ 6513.097767] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6513.730336] LustreError: 193402:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6513.810227] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6518.707889] Lustre: DEBUG MARKER: oleg221-server.virtnet: executing set_default_debug -1 all [ 6519.270691] LustreError: 193403:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6524.386977] LustreError: 193402:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6526.899319] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6527.180605] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 6531.391513] Lustre: DEBUG MARKER: oleg221-server.virtnet: executing set_default_debug -1 all [ 6539.375976] Lustre: DEBUG MARKER: oleg221-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 6545.595349] Lustre: DEBUG MARKER: oleg221-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 6552.436450] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6552.704900] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 6558.853162] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:67 to 0x280000401:97) [ 6559.321317] Lustre: DEBUG MARKER: oleg221-server.virtnet: executing set_default_debug -1 all [ 6566.791356] Lustre: DEBUG MARKER: oleg221-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 6574.138152] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6574.277492] LustreError: lustre-OST0001: New target registration disabled. Use 'lctl set_param allow_register=unlimited' to allow new target registrations. [ 6574.290176] LustreError: lustre-OST0001: the MGS refuses to allow this server to start: rc = -13. Please see messages on the MGS. [ 6574.304908] LustreError: 196227:0:(tgt_mount.c:2617:server_fill_super()) Unable to start targets: -13 [ 6574.316429] LustreError: 196227:0:(tgt_mount.c:2132:server_put_super()) no obd lustre-OST0001 [ 6574.319396] LustreError: 196227:0:(tgt_mount.c:116:server_deregister_mount()) lustre-OST0001 not registered [ 6574.345337] Lustre: server umount lustre-OST0001 complete [ 6574.347587] LustreError: 196227:0:(super25.c:179:lustre_fill_super()) llite: Unable to mount : rc = -13 [ 6578.131849] Lustre: DEBUG MARKER: conf-sanity test_50c: @@@@@@ FAIL: Unable to start OST2 [ 6587.126876] Lustre: DEBUG MARKER: == conf-sanity test 50d: lazystatfs client/server conn race ========================================================== 14:30:19 (1788805819) [ 6595.587311] Lustre: DEBUG MARKER: conf-sanity test_50d: @@@@@@ FAIL: Unable to start MDS